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Summary 


This report documents the results of a study which was conducted in order 
to establish a framework for the quantitative description of the uncertainty in 
measurements conducted in the National Transonic Facility (NTF). The 
importance of uncertainty analysis in both experiment planning and reporting 
results has grown significantly in the past few years. Various methodologies have 
been proposed and the engineering community appears to be "converging" on 
certain accepted practices. The practical application of these methods to the 
complex wind tunnel testing environment at the NASA Langley Research Center 
was based upon terminology and methods established in ANSI and ASME 
standards. The report overviews this methodology. 

Computers have established a dominate role in experimental data 
acquisition and processing in wind tunnels. This study focused on the influence of 
computer based data acquisition in both the calibration process and the actual 
measurement. 

Since many of the instruments used in the NTF are calibrated on a regular 
basis, the uncertainty associated with these calibration experiments was also 
considered. This assessment is complicated by the fact that most of these 
calibration experiments are performed in an environment which is quite different 
from that which the instrument is exposed to in the wind tunnel. The role of the 
calibration experiments and the uncertainty in their results is also discussed in 
this report. 

Preliminary estimates of both bias and precision errors were performed 
using data collected at the NTF. This required a detailed description of the 
measurement process as performed at the NTF and this description is included in 
the report. These preliminary uncertainty estimates were developed to 
demonstrate the methodology for a complete system uncertainty analysis and 
were not intended to provide a comprehensive assessment of the system 
uncertainty. This preliminary study did highlight the importance of certain 
instruments and role of the calibration experiments performed prior to actual 
measurements in the NTF. 
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Section 1 . Introduction 


In recent years there has been an increased awareness of the importance 
of "quality" in every phase of engineering activity. One indication of quality in an 
experimental measurement is the uncertainty or "error" associated with the 
"result" of the experiment. Quantifying the "error" and the associated "level of 
confidence" in a particular result is the purpose of an "experimental uncertainty 
analysis." 

1.1 Background 

Probably the most difficult aspect of performing an "uncertainty analysis" 
for an engineer is that there is no single, well defined methodology which is 
universally accepted. One of the earliest cited references to a systematic 
methodology for performing an uncertainty analysis was developed by Kline and 
McClintock 1 * This early work appeared to provide a "standard" for the 
propagation of error into an experimental result and has been cited in numerous 
textbooks on experimental methods during the past forty years. In the sixties a 
couple of books appeared which provided a brief background in the appropriate 
methods of statistical analysis as applied to the interpretation and presentation of 
experimental data 2,2 and suggested a number of statistical concepts which could 
be applied to the analysis of experimental data. 

As computers began to play a larger part in experimental data processing 
about three decades after the original work of Kline and McClintock, there 
appeared to be increased interest in the issues related to uncertainty analysis as 
indicated in References 4-7. These changes have also lead to a number of more 
recent or revised books specifically dedicated to the subject.^' 1 2 These works 
introduced additional concepts and in particular a growing distinction between 
systematic or bias "errors" and random or "precision" errors and the role that 
each plays in an experimental uncertainty assessment. 

There has also been considerable effort expended, particularly by the 
metrology community, in the application of statistical methods to defining the 
uncertainty in measurement standards for use in instrument calibration and that 


t (All references cited in the text are indicated with superscript numerals and are given in Section 
7. The cited references are not intended to be an all inclusive list of related publications but were 
those that the author found useful in this study). 
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work has continued to the present. 1 1 ‘ 1 5 These efforts appeared to have 
provided a strong foundation for the current statistical methods used in 
uncertainty analysis and in particular issues related to "cun/e fitting." 

As each of these efforts added to the body of knowledge in this area, there 
still was no consensus as to notation or even interpretation of the results of an 
uncertainty analysis. This lack of consensus was eased somewhat by the 
development of an engineering standard in 1985. 16 This standard provided a 
basic framework for the application of uncertainty analysis methods to 
experimental measurements. It also appears to have provided the motivation for 
a very useful textbook 17 which provides additional insight into various terms and 
concepts developed in the Reference 16 and emphasizes the utility of performing 
an uncertainty analysis at various stages of experiment planning, debugging and 
during the presentation of results. 

Where each of the references cited above were "generic" in nature and 
were intended for a wide variety of scientific and engineering applications, the 
aerospace community has also been involved with the development of methods 
and standards with very specific applications in mind. Much of this effort 
appeared to be focused in the gas turbine engine community and a sampling of 
this relatively extensive work is included in References 18-20. 

The importance of uncertainty analysis in experiment planning and 
presentation of results has not been lost on the wind tunnel testing community 
and AGARD is currently involved in the development of a rather comprehensive 
document related to uncertainty analysis for wind tunnel applications (This work 
has not yet been published but a draft was available to the author and will be 
referred to as the "AGARD Draft" in this report). There have also been efforts to 
consider individual aspects of the problem of uncertainty analysis for wind tunnel 
applications, 2 1 but to date this author has not identified a published, compete 
system uncertainty assessment for a wind tunnel. 

A review of the selected references cited above will illustrate the diversity 
in terminology and various concepts that have been applied to this topic called 
uncertainty analysis. Though "standards" have been developed in recent years, 
there still appears to be considerable discussion as to the appropriate methods 
and interpretation of results. This is most likely due to the fundamental character 
of uncertainty analysis which has its basis in the ’inexact" sciences of probability, 
statistics, experience and professional judgment. This is to say that the methods 
used in uncertainty analysis would not be classified as "deterministic" in 
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character and they are often susceptible to considerable confusion; much of the 
confusion is based in semantics. Though the mathematics can be relatively 
straightforward, the semantics can be a problem and the assumptions are 
critical. The best solution to this problem therefore appears to require one to 
apply those techniques which appear to be most appropriate for the particular 
experiment at hand and then to carefully explain how the results have been 
achieved and how they should be interpreted. That has been the approach taken 
in the current effort. 

1 .2 Project Goals 

The purpose of the project described in this report was to build upon 
recent developments as related to uncertainty analysis and to apply them to the 
problem of wind tunnel drag coefficient measurements. Considering the 
limitations imposed by time and resources for this effort, its primary purpose was 
to provide guidelines and a framework within which more detailed evaluation of 
specific wind tunnel test measurements could be conducted in the future. 

The project goal was to initiate the development of a complete, end-to-end 
system accuracy assessment for data obtained in a "production" wind tunnel 
environment. Due to the complexity of the test facility and the cost of its 
operation, this type of wind tunnel is usually highly automated and both the tunnel 
process control and data acquisition are computer-based. It is the interaction 
between the tunnel and its simulation of the "aerodynamic environment", the 
instruments and the data acquisition system which complicate the uncertainty 
analysis for the complete system. Though "accuracy" information is often 
available for individual instruments, this information is often the result of 
calibration experiments which are conducted using'methods and in environments 
much different from the conditions encountered in the wind tunnel. Developing an 
understanding of how the individual instruments are integrated into a complete 
system and how this integration influences the overall system accuracy was one 
of the primary goals of this study. 

Though methods which are applicable to a variety of wind tunnel facilities 
were evaluated and developed as part of this study, the initial application was to 
the problem of drag measurements conducted in the National Transonic Facility 
(NTF) at the NASA Langley Research Center. The purpose was to provide 
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substantiation for a statement of the results for a given test point that the 
measured drag coefficient was: 


Cd = X ± Ux ( with stated confidence level) 

where the uncertainty in the measured value X was quantified as U x . The 
estimation of U x and the confidence level was to be based upon currently 
accepted methods in uncertainty analysis. The ANSI/ASME standard 16 was 
used a guideline for most of the methodology and terminology (though not 
symbology) as applied in this study. The specific application did require the use 
of alternative approaches in certain cases and these exceptions to this standard 
are discussed in the report. 

Section 2 of this report includes a very brief overview of selected topics 
related to the uncertainty analysis presented in this report. It is not the intention of 
this document to provide a general "tutorial" on uncertainty analysis but to 
highlight those issues which have direct bearing on the problem at hand. A more 
comprehensive background in uncertainty analysis can be developed from a 
number of the cited references. 

Section 3 presents a rather detailed description of the "measurement 
process" as it is performed at the NTF. This includes a description of the various 
"calibration" experiments and the “flow" of information from the fundamental 
measurements to the computed results. 

One characteristic of the experiments conducted in the wind tunnels at the 
NASA Langley Research Center is the extensive use of specialized instruments 
and the subsequent requirement for calibration of these instruments. An 
approach for dealing with instrument calibration and the influence of experimental 
uncertainty in the calibration experiment on the final measured result is presented 
in Section 4. 

In order to demonstrate the techniques presented in this report a brief 
example is provided using data from a test in the NTF and this is presented in 
Section 5. Most of the discussion is related to two test points for a subsonic 
transport configuration operating at relatively low angles of attack. The data 
reduction procedures used in the NTF were employed to provide much of the 
information required to make the uncertainty estimates. Both bias and precision 
estimates are made for these test points and combined to provide an estimate of 
the experimental uncertainty. The methods used to estimate each contribution to 
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the uncertainty are presented for this example. Due to limitations on time and 
expertise, only a limited number of potential contributions to the system 
uncertainty could be considered during this preliminary study. Issues such as 
modeling of the wind tunnel gas, wind tunnel corrections and model dynamics 
have not been included in the uncertainty estimates. It should be emphasized 
that these preliminary estimates were developed in order to establish a 
framework for more comprehensive uncertainty analyses to be performed in the 
future. 

The final section, Section 6, provides a summary of some of the 
observations and conclusions as well as issues to be considered as this 
preliminary uncertainty assessment is extended and applied to future wind tunnel 
tests. 

One final note of introduction to this study. Much of the methodology 
associated with uncertainty analysis is "uncertain." The reader should be 
cautioned that in order to estimate the uncertainty in an experimental result, there 
are many assumptions which must be made and considerable engineering 
judgment exercised - an aspect that is upsetting to some more "scientifically" 
oriented engineers. Though there is some mathematics involved which appears 
to provide a "quantitative" foundation for the estimates, the success of the 
analysis will depend upon sound understanding of the measurement process, as 
well as experience and one's ability to make good "guesses". The approximate 
nature of the results should always be considered and one should avoid getting 
overwhelmed by minute details as they proceed toward the overall goal. 
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Section 2. Overview of Experimental Uncertainty Methodology 

The previous Section cited a series of references which describe various 
concepts and methods associated with experimental uncertainty analysis. The 
following discussion is not intended as a comprehensive review of uncertainty 
analysis but as a brief overview of various issues of interest in this study. Many of 
the concepts and much of the terminology has been adapted from References 16 
and the "AGARD Draft". 

2.1 Basic Concepts 

Measurement uncertainty analysis can be used to plan an experiment, 
identify corrective action in order to achieve test objectives, or to qualify the 
results of an experiment. Depending upon the purpose of the analysis different 
information will be available and different procedures may be followed 17 . In the 
reporting phase of the project, and that is the phase that this study is concerned 
with, the purpose of the uncertainty analysis is to determine numerical estimates 
to provide upper limits to, 

1 . random precision errors and, 

2. systematic or fixed bias errors. 

These two components of the total measurement error are then combined to 
provide an estimate of the uncertainty in the results of the measurement. These 
two components will be described in more detail in the following section but it 
should be emphasized that radically different approaches are taken to determine 
each component. The precision error estimate is usually based on a statistical 
evaluation of the results of numerous experiments and is in some respects a 
measure of the repeatability of the measurement process and inherent 
unsteadiness in the phenomena being studied. The bias error estimate is truly an 
"estimate" and has its basis in experience and engineering judgment - which can 
create problems for an engineer who wants to be “certain about the uncertainty." 

The final issue that must be addressed is the amount of information 
available on the measurement process in order to perform the uncertainty 
analysis. For the current study it was assumed that the measurement process 
was completely defined and the methodology established. In the case of the 
NTF, the procedures for data acquisition and processing have evolved during the 
past 10 years and there are well established testing techniques. In the process of 
acquiring and processing the information necessary to develop the "result" of the 
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measurement all known errors have been eliminated and all known calibration 
corrections have been applied. Since the measurements that will be evaluated 
have already been performed, some insight is available from previous tests as 
well as the experiences of instrumentation and wind-tunnel test engineers. All of 
the equipment and instruments used in the measurement have either been 
calibrated or the manufacturer's specifications are available. 

2.2 Terminology 

Various terms are used to describe the accuracy, uncertainty, precision, 
quality... etc. of the results of an experiment or a "measurement". Often each 
term carries with it numerous concepts and the use of a particular term may be 
confusing if it is not carefully defined. It was the author's experience that 
establishing a common framework for the discussion was often the most crucial 
step in arriving at a useful result from an uncertainty analysis and conveying that 
result to others. This is due to the fact that certain aspects of the process called 
"uncertainty analysis" are still evolving. Many of the references cited in the 
previous Section are primarily devoted to establishing a "semantic" framework for 
discussing measurement uncertainty and should be studied in detail by anyone 
who wishes to become conversant in this language. It is the responsibility of the 
engineer who attempts to quantify the uncertainty associated with a stated result 
to carefully define the approach taken and the terms used. 

The following is very brief collection of a limited number of terms or 
concepts which are used in this report. These definitions are not meant to be all 
inclusive but are intended to help the discussion in this report. 

measurement or measured parameter - a useful, quantified parameter, this may 
be bits (i.e. digital representation of an analog voltage) , volts, temperature, force, 
etc.. This "number" is often used in subsequent calculations to determine a 
"result". 

result - a quantity determined from the numerical manipulation of individual 
measurements and other numerical quantities such as handbook values, data 
from tables, etc.. 
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measurement error - difference between an "estimated" value (be it a 
measurement of a single parameter or a result) of a quantity and the "true value" 
of the quantity. 

true value - the actual value of the parameter being measured (at the instant of 
the measurement) . It is important to realize that the true value is always an 
"unknown" and its definition depends upon how the measurement will be used in 
a "conceptual" sense. There are numerous variations on the concept of "true 
value" and each may have an appropriate place in a given experiment. 

fixed bias error - systematic error which is "constant" for some known period of 
time (e.g. the error in a calibration constant for a particular instrument which is 
fixed between calibrations). 

random precision error - probabilistic component of the error which is due to 
variations, often temporal, in the measurement system characteristics or the 
process being measured (e.g. the variations in local flow angle due to wind tunnel 
turbulence). 

precision limit- an estimate, with statistical support, of the "limits" on the 
precision error with a statement of confidence. 

bias limit - an estimate of the "limits" on the true bias error with an assumed or 
inferred statement of confidence. 

confidence level - a quantifiable expression indicating the probability that the true 
value is within the stated limit of the estimated value of a quantity. 

measurement uncertainty - a combination of the precision and bias errors and its 
associated confidence level. 

2.3 Measurement Error Sources 

All experimental measurements (not to be confused with counting 
experiments) have some error associated with them. Often this error does not 
influence the utility of the measurement but in some cases quantifying the error 
and understanding its origin can add considerably to the value of the result. In the 
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wind tunnel experiments considered during this study there were various sources 
of measurement error. Classifying the sources of these errors, as suggested by 
many of the practitioners of uncertainty analysis 1 6-1 7, allows for a systematic 
assessment of their influence on the measurements. A useful, though not unique, 
classification of errors includes calibration errors, data acquisition errors and data 
reduction errors. 

One important aspect of the NASA wind tunnel test environment is the role 
of calibration experiments. All of the instruments and sensors used to monitor the 
tunnel and to measure the model orientation and aerodynamic forces are 
calibrated on a regular basis. These calibration experiments are used to monitor 
the condition of the instruments and sensors and to provide "calibration" 
constants to characterize the performance of the instruments. The calibration 
experiments also have uncertainty associated with their results. The uncertainty 
associated with the results of these calibration experiments influences the results 
achieved in the wind tunnel tests since the calibration data is used to process 
measurements made with the same instruments and sensors in the wind tunnel. 
The calibration errors can include the errors associated with the working 
standards used as part of the calibration, the manner in which the calibration 
experiment is conducted and the data processing or "curve fitting" which used to 
provide the parameters which characterize the instrument’s performance. It 
should be emphasized that "between calibrations" many of the errors associated 
with the calibration process are fixed since the calibration parameters do not 
change. Since this calibration information is "fixed or frozen" between instrument 
calibrations these errors contribute to the systematic or bias errors in the result. 
Early in the development of the methods of uncertainty analysis it was often 
stated that calibration '"removed" systematic errors and though calibration may 
serve to reduce systematic errors, it cannot completely eliminate them. The 
importance of the calibration experiments and their associated uncertainty is 
discussed in more detail in Section 4 of this report. 

The data acquisition process is also a source of measurement error. In the 
modern wind tunnel environment the data acquisition process is composed of two 
basic steps. The first is the conversion of a physical property (i.e. temperature, 
pressure, force, etc.) to a proportional analog equivalent in the form of a "voltage 
using an electronic transducer. The second step is the measurement of voltage 
by a computer-based, electronic analog to digital (A/D) conversion processor. 
Thus the basic "measurement" is the conversion of analog voltages to "bits". 
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All of those factors which influence the generation of the voltage signal 
which originates at the transducer or sensor, the relationship between the desired 
physical quantity and the voltage and everything that influences the voltage 
before the A/D conversion may be classified as data acquisition errors. This 
would include instrumentation error sources (i.e. voltage supplies, filters, 
amplifiers, ..), environmental effects (i.e. temperature, humidity,..), sensor and 
probe errors, spatial errors and others particular to a given system. These data 
acquisition errors can contribute to both the bias and precision components of the 
uncertainty. Only through a complete understanding of the measurement system 
and the phenomena being studied can these error sources be effectively 
identified. It appears to this author that the most important step in a uncertainty 
analysis is developing a complete understanding of the measurement process. 
Without this the uncertainty analysis will be of limited value. 

The third classification of error sources are related to data reduction. Once 
the results of the calibration experiments have been "frozen" as calibration 
constants and the voltages converted to "bits" the remaining processes to be 
performed in order to achieve the result of an experiment are the selection of 
appropriate "physical constants" and numerical calculations. Often the data 
reduction process is similar to that used in the calibration experiments to provide 
the parameters for subsequent measurements with the instruments. 

Computational resolution, interpolation, iterations, curve fitting and other 
numerical procedures can be the sources of data reduction errors. 

As will be discussed in the following sections, the "error" in a 
measurement, be it either bias or precision, will be the result of various error 
sources. The ability to identify the appropriate error sources and quantify their 
contributions represents the basis of an uncertainty analysis. In the current study 
only limited time and experience was available to the author in the attempt to 
identify potential error sources. One of the most important outstanding issues 
remaining at the completion of this preliminary uncertainty analysis is to perform 
a more comprehensive assessment of the error sources particular to the NTF. 

2.4 Uncertainty Analysis Methodology 

As indicated earlier, the methodology outlined in Reference 17, with some 
exceptions, was adapted for the current study. The following six steps serve as 
an outline for the procedure followed in this study. 
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1 . Define the measurement process 

2. Identify and quantify the elemental bias error sources 

3. Calculate bias errors for each measured parameter 

4. Propagate bias errors through the data reduction process to the result 

5. Estimate the precision limit of the result from multiple measurements 

6. Calculate "uncertainty" of the result 

These steps can be illustrated schematically as shown in Figure 1 . This 
figure, which was adapted from similar "flowcharts" in Reference 17, illustrates 
the "flow" of error sources from the individual measurement systems and 
individual measured parameters to the result. Actually two different approaches 
for quantifying the precision error are considered in this report. If appropriate data 
is available estimates of the precision error from multiple measurements of the 
result is preferred, although not always possible. 

Since the purpose for the study was to evaluate the uncertainty in the drag 
measurements in the NTF one of the first steps was selecting a set of data 
associated with an already completed test program which could be used to 
evaluate the methods which were developed. Fortunately data existed which 
could be used to quantify the uncertainty using either single sample or multiple 
measurement approaches. Using the terminology of References 16 and 17, the 
preferred procedure, outlined in Figure 1 , would be based on "more that one" test 
or be referred to as an Nth order uncertainty analysis with an end-to-end 
assessment of the precision error. The following sections provide some insight 
into the details for each of these steps and the preliminary test case is presented 
later in the report. 

2.5 Precision Error 

The precision error (or often referred to as the repeatability) estimate has 
its basis in probability and statistics. The "scatter" in data observed when multiple 
measurements are recorded at nominally the same test conditions provides a 
qualitative assessment of the precision of the measurement process as well as 
an indication of the variations occurring in the process being measured. 

Extracting useful quantitative insight from these multiple measurements can be 
accomplished through statistical analysis of the information. References 22 and 
23 provide a good background in many of the concepts which are used to define 
the precision error in an estimate. They should be referred to for additional details 
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and the following is included to provide as only a brief overview of certain issues 
central to the current application to uncertainty analysis. 

Consider a single number, X-), which represents an individual 
measurement or realization of a parameter, X. If that measurement is repeated 
and yields X 2 , one might expect a different result from the original measurement. 
This can be done many (N) times. The most complete representation of the 
statistical character of the variable X would be if N was allowed to go to infinity. 
Two parameters that define the "characteristics" of an infinite population of 
random numbers are the mean, p, and standard deviation, a. 


1 N 
N I X i 

N — » °o i = 1 


a = lim 

N — > °o 


1 

N 



2 

Xj-p) 


1 

2 


Eq. 2.5.1 


Eq. 2.5.2 


In the case of experimental data, the mean can be interpreted as a "best 
estimate" of the parameter and the standard deviation is an indication of the 
"scatter" in the data. The utility of these statistics often depends upon the 
probability distribution of the data and the following discussion assumes that the 
random variables of interest have a normal or Gaussian probability distribution. 

Unfortunately one must work with a "finite" number of samples which often 
only represent a small subset of the "parent" population. One attempts to 
generalize to the manner in which the random numbers are "distributed" in the 
parent population from the limited sample. The size of the sample influences the 
ability to extrapolate to the parent population. The finite sample size requires the 
definition of finite sums ( the infinite sums indicated above would be very difficult 
to evaluate since it may take a long time to get that much data!) The mean of a 
sample of a population is written as, 

- 1 N 

X = N I X i Eq. 2.5.3 

k = 1 
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This is an estimate of the mean of the parent population and it depends upon the 
sample size, N, and the statistical characteristics of the data. An "estimate" of the 

standard deviation of the population is referred to as the precision index of the 
sample, S x . 



k = 1 


With one additional piece of information the two estimated parameters, 

X and S Y , can then be used to make "probabilistic" statements about the mean 

A 

value and "future" measurements. 

For a normally distributed random variable one can make certain 
statements about the probability that a single "new" realization of the random 
variable will lie within some prescribed interval about the mean. For relatively 
large sample sizes (and thirty is large enough in most cases) it can be shown that 
approximately 68% of the time a "new" random number will fall within a range of 

± S of X. It is also true that 95% of the time this "new" value will be a range of 

X 

±2S X of X . This provides very useful information particularly if you recall that the 

goal of the uncertainty analysis is to identify "intervals" in which true values of the 
measurements lie. This information not only provides the size of the interval but 
also allows one to state the "confidence or coverage" placed on these interval as 
indicated by the percentage of times that a random variable would fall in the 
interval. 

Recall that the approximation to the population standard deviation 
depends upon the size of the sample. As the sample size increases, the 
approximation is improved. For sample sizes less than thirty, the value of S x can 
be scaled by a "t" parameter to allow for similar statements to those made above. 
The sample size is used to introduce one additional concept useful in statistics 
and that is the "degrees-of-freedom" which is related to the size of the sample 
and certain characteristics of the statistic derived from the sample. Though this 
term is used in many statistical inference techniques its use in the current 
application is limited as will be discussed below. Based upon the sample size it is 
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possible to select a scale factor "t" for a selected value of confidence level, C , 
so that one can make the statement for a single realization of the random 
variable Xj : 


Xj - 1 S x < p < Xj + t S x C% of the time " 

The values for "t" for a desired confidence level, C, and sample size, N, can be 
found in tabular form in most books on statistics or estimated using rather 
straightforward numerical techniques 24 . In this way one can make a statement 
about the mean of the population, often considered the true value for an 
unbiased parameter, in terms of a single measurement and the precision index of 
the sample. The interval width, t S x , introduced above is referred to as the 
precision limit, P x . In uncertainty analysis the precision limit is an estimate of the 
precision error for a given confidence level. 

One additional concept which is useful .when the mean value of multiple 
measurements is used to estimate the true mean value, is the precision index of 

the mean. Precision index of the mean or the sample standard deviation of the 
mean, S- , 



can be used in conjunction with the appropriate t factor to indicate the interval in 
which the population mean lies with respect to the sample mean. This allows one 

to make a similar statement as that above for the mean of the sample, X 


" X -t 


S x 

vr " 4 


<x + t 


A 

Vn 


C% of the time 


Reference 16 goes to great lengths to discuss various alternative methods 
for estimating the precision index and its associated degrees-of-freedom for 
cases where there are multiple sets of measurements available or samples of 
various sizes. In the AGARD Draft, the recommendation is made that for most 
practical cases related to wind tunnel applications making the assumption of 
large sample sizes and assuming a t factor of 2 and a coverage of 95% will 
provide useful results and eliminate much the complexity associated with the 
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determination of the precision error. This is just an example of a lack of a 
universally accepted approach and one must just be careful to explain how the 
issue of sample size and coverage was treated in a particular uncertainty 
analysis. 

One other issue that must be addressed when determining the precision 
index for a sample is that the information must be sampled over an 'appropriate' 
time interval to adequately represent the precision index. Consider the time 
history of the ’'data" shown in Figure 2. If the data is sampled during the time 
interval At shown in Figure 2a, an incorrect representation of the precision index 
will be determined. This becomes a problem in many of today's computer based 
data acquisition systems where large amount of data can be recorded in a very 
short period of time. This is an important issue when one wants to make multiple 
measurements and compute the mean value of a parameter so that the precision 
error estimate can be determined using the precision index of the mean. The 
implication being the larger the "N," the smaller the precision index, and this can 
lead to erroneously small values of the precision error. 

In a similar manner the sample must include all of the sources of precision 
error in the experiment that one desires to consider in assessing the uncertainty 
in the experimental result. The concepts of precision and repeated 
measurements are directly related and the manner in which the experiment is 
repeated influences the contribution of each factor. In the case of wind tunnel 
tests one may wish to consider the influences of test condition unsteadiness, the 
ability to duplicate test conditions or model positions, the influence of model 
assembly and disassembly and dynamic effects such as model vibration. Thus 
simply making multiple sequential measurements at a single "test point" may not 
allow one to adequately characterize the precision in a given experiment. 

The discussion above implied that the calculations were being performed 
in order to determine the precision in a desired "result" by conducting multiple 
experiments to yield that result. That would be referred to as an end-to-end 
precision error estimate and if that information is available it should be the 
preferred method for making an estimate of the precision error. As indicated in 
many of the previously cited references, it is also possible to determine the 
precision index for each of the measurements that are used to determine a result. 
If this is done then the precision error of the result can be determined using the 
propagation of errors approach identical to that to be discussed in the next 
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section on bias errors. The selection of the appropriate approach is usually based 
upon the availability of appropriate data. 

2.6 Bias Error 

Unlike the previous discussion of precision error where it was 
recommended that the precision error be determined on an "end-to-end" basis 
and was "statistical" in nature, the bias error estimates must be done at the level 
of the individual parameter measurements and is based more upon judgment 
than arithmetic. The fundamental character of bias errors is that they are "fixed" 
and do not vary as additional measurements are made as one attempts to repeat 
the experiment. They represent the difference between the true mean of the 
population of all possible measurements and the true value of the desired 
parameter. Since neither the true mean nor the true value are known, estimating 
the bias error is often quite difficult. 

Figure 1 schematically illustrates the manner in which the bias errors are 
estimated. Individual bias contributions are identified and quantified for each 
parameter used in the determination of the result. These bias errors are 
combined to provide a bias limit for each parameter. The bias limit is an estimate 
of the upper limit of the bias error and it is stated with a confidence level or 
"coverage" consistent with the desired overall uncertainty. This coverage should 
be consistent with that used to describe the precision limit. As before one wishes 
to make an estimate of the bias limit, B p , for a given parameter P such that , 

P - Bp < P TRUE < P + B p C % of the time " 

The bias limits are not influenced by "repeated" measurements. 

The procedure for estimating the bias limits first requires the identification 
of each parameter which is used to "compute" the result. Some of these 
parameters are measurements conducted as part of the given experiment, others 
are the results of earlier experiments or calibration experiments and still others 
are taken from handbooks, plots, etc.. Once each parameter is "fixed" as a 
"number" with a finite number of significant figures the bias error in each will 
contribute to the bias error in the final result. 

In order to identify and subsequently quantify the bias error contributions 
to each parameter it is often helpful to separate them into various classes of error 
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sources. The same classification discussed above can be used to help identify 
potential bias error sources. 

1 . Calibration errors - depends upon the details of the calibration process 

2. Data Acquisition / Installation errors 

3. Data Reduction errors 

The value of each of these bias contributions may depend upon the value of the 
parameter so the bias limit can vary with test conditions. Once all of the potential 
contributions to the bias limit for the parameter P have been identified and 

quantified they are combined by addition in "quadrature". The expression , B p ^ 

represents the j th bias source in the i th error classification. Then the total bias 
limit in the parameter P due to each elemental bias contribution is estimated as, 

2 

Eq 2.6.1 

This implies that all of the contributions would not be expected to occur 
simultaneously and, characteristic of quadrature addition, the final bias limit is 
dominated by the most significant terms. 

It should be noted that bias errors need not be symmetric. They can be 
can be asymmetric ( i.e. Bj j (+) ^ Bj j (-) ) and then each "side" of the bias limit 
must be determined independently. This was not an issue in this study and the 
reader is referred to Reference 17 for a more detailed development of 
asymmetric bias limits. 

It was this author's experience that the most difficult task in performing the 
uncertainty analysis for the NTF was estimating the bias limits for the parameters 
considered in this study. A number of approaches were taken and they are 
discussed in the Sections of this report related to calibration and results. The 
usefulness of the final results obviously depends upon one's ability to identify 
those critical components of the bias error and to establish reasonable values for 
each. 
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2.7 Uncertainty in a Result 


The result of an experiment is often a function of a number of 
independently measured parameters and "constants" which were the results of 
earlier experiments (often calibration experiments) or analysis. The result can 
often be expressed as an analytic function "r“ of several variables, Pj , and thus 
can be written as a closed-form expression for the result, 

r=r ( p i . p 2 ’ > p j ) Eq 2.7.1 

In some cases the result may be developed using a computer based numerical 
algorithm which cannot be written as a simple analytic expression. The 
uncertainty in the result computed by either an analytic expression or a numerical 
algorithm will be the result of the uncertainty in each of the parameters used to 
determine the result and any errors" associated with the numerical or analytic 
procedures used to compute the result. 

The method used to estimate the error in the final result, referred to as the 
propagation of errors, is based upon a first order Taylor series expansion of the 
result in the region near the nominal value of the result. A somewhat detailed 
development of the basic expression for the propagation of errors in presented in 
an appendix of Reference 1 7 and in a number of the other previously cited 
references. It is again important to emphasize that when one determines the 
uncertainty in a result that it is an approximation. The value of the uncertainty in 
the result in most cases is only valid for the nominal values of the parameter Pj 
which were used to compute the result. Implying that a value of uncertainty 
determined for a single test point is valid throughout the operating range of an 
experiment can lead to erroneous conclusions. 

As indicated in the earlier discussion, in the reporting phase of an 
experiment the best estimates for the precision limit of the results would be 
determined using a statistical assessment of multiple measurements under 
appropriately controlled, "repeated" conditions. That was the approach taken for 
the preliminary results developed as part of this study. Therefore the emphasis in 
this section is related to the propagation of bias errors into the computation of the 
results of an experiment. 

The expression for determining the bias limit for a result r is determined 
using a relation in the form, 
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Eq. 2.7.2 



X (0 i B Pi > 2 


i = 1 


where 0j are the "sensitivities" of the result to variations in each parameter Pj: 

The sensitivities are first partial derivatives, or approximations thereof, of the 
result with respect to each parameter. If the analytic expression for r can be 
formulated in a reasonable fashion then, 


dr 

9i = ^ 


Eq. 2.7.3 


This expression would then be evaluated at the appropriate nominal values of the 
parameters, Pj. If the expression for r is not readily available or performing the 
differentiation is not straightforward, as is the case in many computer-based data 
reduction procedures, a difference numerical approximation can be developed for 
the partial derivative, 


Ar 

6 ' “ AP: 


r ( P i , P 2 > P j +AP j ... , Pj ) - r( P 1 , P 2 , Pj- , Pj ) 


AP : 


Eq. 2.7.4 


Equations 2.7.3 and/or 2.7.4 can be used in conjunction with Equation 2.7.2 to 
estimate the bias limit for the result r. An alternative set of expressions which 
may also be used provides the "relative" bias in the result based upon relative 
bias limits for each of the contributing parameters and relative sensitivities. One 
advantage in dealing with the relative formulation is that it can help avoid some 
problems with "units". The expression for the relative bias limit is, 
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Eq. 2.7.5 


Where the relative sensitivities, 0:' , are defined as, 


0 ;' = 


3r / r 


P: 


dr 


' 5P i /P i 


r 0P: 


for cases where the analytic derivatives can be determined, or 


Eq. 2.7.6 


0' = - Ar/r 
1 APj/Pj 


Eq. 2.7.7 


for the numerical approximation. Again each sensitivity is computed at "nominal" 
values of individual parameters, Pj . 

The expressions given above are for those cases in which the bias in each 
of the parameters Pj are independent, that is they are uncorrelated. For example 
consider a case where two pressure transducers are calibrated using the same 
working standard. The contribution to the bias limit for each individual pressure 
sensor due to the working standard would be "perfectly" correlated for each 
measurement. If the bias limits for some of the parameters used in computing the 
result are correlated then additional terms are needed in the expression for the 
bias limit of the result. It takes the form, 
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Eq.2.7.8 


where 6 jk is the Kronecker delta ( = 1 if i = k and =0 if i * k ) and p Bjk is the 
correlation coefficient between the biases B p . and Bp k . The issue of correlated 

bias limits is addressed in more detail in Reference 17. Appendices A and B have 
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been included in this report to provide additional insight into the problems which 
can be encountered when determining the bias limit in a result. 

Once both the bias limit, B r , and the precision index, S r , and associated "t" 
factor for the result have been determined for a particular test point or set of test 
conditions, there are two options for stating the uncertainty in the result 
depending upon the manner in which the bias and precision errors are combined. 
The two options influence the "coverage" or level of confidence of the resulting 
uncertainty. Note that finally after using the terminology "error" up to this point in 
the development, the term "uncertainty" is now applied to the final combined 
result. If the bias limit and precision limit are simply added or "superimposed" 
then one can write, 

%dd = (b ' +ts ' ) - Eq - 2 - 7 ' 9 

If the bias limit was comparable in coverage to the precision limit , and the 
precision limit was based upon a 95% coverage, then Eq. 2.7.9 (i.e. simply 
adding the two components) provides an uncertainty in the result with 99% 
coverage or the "odds" are 99 in 100 that the true value of the result with lie 
within this interval of the computed result. 

It appears as if the preferred approach for combining the bias and 
precision errors is addition in quadrature, much like that used to combine the bias 
limits. This takes the form, 


U 


r RSS 



Eq. 2.7.10 


and if the "t" factor is selected for a 95% coverage, the uncertainty in the result is 
also then determined for 95% coverage. In this report Eq. 2.7.10 is used to 
compute the uncertainty in the result. This the allow a statement for the result in a 
form, 


r ± U r (95% confidence level) Eq. 2.7.1 1 

Once the uncertainty in the result has been established it is possible to more 
effectively evaluate how the result can be interpreted and used in future 
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calculations. Some sources indicate that it would be preferred to state the 
uncertainty as well as the bias limit, precision index and associated degrees of 
freedom so that when this result is used in other computations one can establish 
their uncertainty. The specific form in which the uncertainty is stated may not be 
as important as providing an associated description of how it was determined, the 
assumptions made, etc.. Until a truly accepted standard for uncertainty analysis 
is established, it is up to the individual to make sure that they have adequately 
supported and documented the approach taken. 
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Section 3. Measurement Process in the NTF 


The National Transonic Facility is a unique national resource capable of 
high-Reynolds-number, wind-tunnel testing. References 25-27 provide details on 
the NTF, its operation and data systems. Test planning, model development and 
tunnel operation in this facility are complicated since it is a cryogenic wind tunnel 
and testing is normally conducted at very high total and dynamic pressures as 
well as very low total temperatures. The complexity of the system and its 
associated costs have provided the motivation for the facility to be continuously 
aware of the importance of data quality and system productivity. 

3.1 Overview of the Data Acquisition System for the NTF 

The facility uses a state-of-the-art computer based, automated data 
acquisition (DAS) and data reduction systems. There are multiple computers and 
various procedures which can be used by the NTF in conducting a wind tunnel 
test. The current report is limited to a simple example in which the test objectives 
would be to measure the lift and drag coefficients, corresponding angle of attack, 
Mach number and Reynolds number for a single configuration. The data for a 
single test condition was recorded and analyzed after the test using off-line data 
reduction procedures. 

As indicated in Section 2.4 the first step in an uncertainty analysis is to 
"describe the measurement process" and that is the purpose of this section. In 
order to determine the drag coefficient for a single configuration at a single test 
condition a number of "experiments" must be performed and information from 
each of these experiments is used to compute the final result, the drag 
coefficient. To provide some perspective for the complexity of this task, there are 
approximately 1 1 "experiments" performed prior to the actual tunnel-on 
measurement and a conservative estimate indicated that there are more than 
250,000 pieces of information which are acquired and processed to make a 
single Cd calculation. 

The following attempts to outline in a very general manner the 
"measurement process" for drag measurements in the NTF. Only limited details 
are provided for each aspect of the process and the goal of the section is to 
identify the primary flow of information and its eventual impact on the system 
wide data uncertainty. More details on the current data acquisition system and 
data reduction procedures should be available by contacting the NTF. 
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A very crude "abstraction" of the NTF measurement system is shown in 
Figure 3. The primary components in the system are the wind tunnel, the 
"aircraft" model, the pressure and temperature sensing systems which monitor 
the tunnel conditions and the force balance and model orientation instruments 
which monitor the model. The information from each of the main instrumentation 
systems is acquired by the data acquisition system (DAU) and converted into 
digital information for subsequent processing by the data reduction software. 

A more detailed representation of the system was prepared and is shown 
in Figure 4. The figure is "inverted" and the information in the system "flows" from 
the bottom to the top. There are a series of sensors used to monitor the tunnel 
conditions and their output is used to compute the desired "results" such as Mach 
number, Reynolds number, dynamic pressure, etc.. The wind tunnel model's 
interaction with the fluid results in pressures acting on the model and a 
temperature distribution within the model. The model is supported by a six- 
component, strain gage balance. The electrical, analog output from the various 
temperature, pressure, orientation and strain sensors is passed through a signal 
conditioning system which independently filters and amplifies each signal. These 
analog signals are then multiplexed, sampled and converted to digital 
representation. All of the sensor input, except the orientation or AOA sensors, are 
digitized by a single A-to-D unit. This information is then combined with a 
significant amount of other "data" as part of the data reduction process. 

A wide variety of results are available each having their own respective 
uncertainty and each additional calculation has the potential for adding to the 
uncertainty. In the case of aerodynamic coefficients, various "corrections" are 
often required in order to transform a non-dimensional "force" measured in the 
wind tunnel to an estimate of the actual aerodynamic characteristic that the full- 
scale aircraft would encounter in flight. The study documented in this report did 
not address many of the issues of data processing but focused on the process up 
through the point indicated by the box labeled "Uncorrected Coefficients". Once 
one has determined the uncertainty associated with the uncorrected coefficients 
it can influence the selection and assessment of the various corrections which 
can be applied to the results. 

A series of pressure transducers are used to sense the total pressure in 
the tunnel plenum and the static pressure in the test section. Currently Ruska 
transducers are used and they are connected via a series of manifolds so that 
the specific transducer which is being monitored by the data acquisition system 


24 


depends upon the pressure level being sensed. This pressure measuring system 
and the transducers are referred to as "Ruskas" through the remainder of this 
report. The temperature measurement system which is used to sense the total 
temperature in the tunnel uses a platinum-resistance temperature probe. This 
probe and the temperature measurement system are referred to as the PRT in 
this report. Additional details on these systems are provided in Reference 27. 

The two central features of the type of measurement system used in the 
NTF are the sensors which convert the physical properties into electrical voltages 
and the process of conversion of the analog information to digital information. 
Many of the issues related to the sensors are far beyond the scope of this limited 
study. Since many of the sensors are "calibrated" the performance of the sensor 
is not directly considered in the uncertainty analysis. Issues related to "is the 
sensor actually sensing what is desired" are central to the uncertainty estimates. 
From the perspective of the test engineer, the process of converting the analog 
signals to digital information is central to the measurement process. One might 
consider that this is where the actual "ngeasurement" takes place. Once this has 
occurred, the information is "frozen" (neglecting certain issues relate to 
computational round-off, truncation, etc.,.) and in the modern computing 
environment, things become quite precise. 

Figure 5 presents a more detailed schematic representation of this part of 
the data acquisition process. Each analog channel, of the three represented in 
the figure, is amplified and filtered before it is introduced to the programmable 
gain amplifier, PGA, which is part of the analog to digital converter. The manner 
in which the analog voltage signal is processed is important to the uncertainty 
analyst since it is the voltage level of the signal, which varies from test point to 
test point, that will influence the relative uncertainty in the digitized result. Also 
each of these components modify the "signal" produced by the sensor. As 
indicated on this schematic each of these channels can be independently 
"calibrated". The calibration of each of these channels, done on a daily basis, 
also influences the uncertainty in the final result. Figure 6 illustrates a more 
detailed view of a single channel of the analog input to the DAU. It is used to 
illustrate the various configurations that this system can assume. The input 
amplifier gain can be varied from 1 to 500. Each channel is filtered at either 1 or 
10 Hz and this has a marked effect on the sampling rates, data averaging and 
the ability of the system to rapidly move from test point to test point. Additional 
amplification can occur after multiplexing and the analog data is converted to its 
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digital representation at rates from 10 to 50 samples per second per channel. For 
the test cases used in this report each channel of analog data was sampled at 1 0 
Hz. 


3.2 Basic Measurement Systems 

The measurement system described above, also known as the NTF, is not 
only used to collect information on the wind tunnel model of interest, but also to 
conduct a number of "calibration" experiments. It is actually only one of a number 
of "facilities" used to develop the information that is necessary to interpret the 
information collected during aerodynamic testing in the NTF. The following 
discussion overviews the individual experimental "set-ups" for the experiments 
required to collect all of the information necessary to measure the drag coefficient 
on a wind tunnel model. It highlights the information collected as a result of each 
experiment. Each of these "set-ups" were used to define each of the steps in the 
measurement process and to highlight for the author those points in the process 
where data was "collected" and information processed. 

3.2.1 DAU System 

One of the fundamental systems is the DAU. It is used to acquire a 
majority of the data collected at the NTF. Therefore one of the basic experiments 
which is required is the calibration of the DAU. Figure 7 illustrates the DAU and 
the process used for calibration. This type of schematic will be used to represent 
each of the basic systems. The "source" of the information is usually presented in 
a box in the upper left hand corner of the figure, information "flows" from the 
source in an analog form (solid lines) or in digital format (dashed lines). The 
output from the process is on the lower right-hand side of the figure in the "bold" 
oval. The "units" associated with the information are often indicated at 
appropriate locations on the figures. Some of the elements in the figures 
represent pieces of equipment such as filters, amplifiers or A/D converters, 
others represent data reduction processes like curve fitting. These figures are a 
useful way of understanding how information progresses from analog to digital 
form and to differentiate between data collection and data processing activities. 
This is an important distinction in understanding the measurement process from 
an uncertainty analysis perspective. 

For the DAU calibration a precision voltage source is used as the working 
standard. The voltage source is "commanded" to produce a certain voltage level 
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which, upon closure of the calibration relay contact, introduces that voltage to a 
given channel of the DAU. For a preselected and commanded voltage level the 
output from the analog to digital converter in bits is recorded. This voltage and 
"bits" information are processed by a linear curve fit to provide a slope and offset 
or bias (not to be confused with a bias limit or error) for each channel of the DAU. 
This slope and bias information is then stored and used to convert bits to digital 
"voltage" information in future measurements. Additional details on the DAU 
calibration process are included in Reference 28. 

As with each of the systems or experiments discussed in the following 
sections, anytime an experiment generates a result that result has an uncertainty 
associated with it. In the case of the DAU the slope and bias for each channel are 
the results of this experiment and in order to eventually perform an uncertainty 
analysis for the complete system it is necessary to define the uncertainty in each 
of these results. Though both bias errors, such as those associated with the 
working standard , and precision errors, such as those associated with variations 
in the calibration experimental process, are present in these results, the result 
itself is very precise and it doesn't vary until the next calibration. If a result of the 
calibration experiment is in error, this error will be propagated as a bias error 
whenever the result is used in future calculations. 

3.2.2 Pressure Measurement System 

There are three separate "experiments" associated with the pressure 
measurement system. Recall that at any one test point there are at least two 
pressure measurements being taken, one static pressure and one total pressure. 
There are a number of possible transducers that could be used depending on the 
pressure level but for each transducer there is a calibration experiment in a 
calibration lab, a "two-point" calibration in the wind'tunnel and then the actual 
measurement conducted at the test point. Figures 8-10 . 

The Ruska is calibrated in a calibration laboratory using a dead weight 
tester as a working standard. The output from the Ruska is digitized using an 
integrating digital voltmeter (IDV). The working standard pressure (psi) and the 
IDV output (volts) are then used to provide calibration slope information. A bias is 
also computed but this is only used as "check" and the actual calibration curve 
offset or bias is determined using the experiment discussed below. 

This highlights a point made earlier, that many of the instruments used in 
the NTF are calibrated in an environment and with procedures much different 
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than those used in the actual measurement process. Therefore the uncertainty of 
the instrument from the calibration experiment may not be the same as the level 
of uncertainty achieved in the actual measurement. 

Figure 9 illustrates a second calibration experiment performed for the 
Ruska. In this case the Ruska is tested when installed in the wind tunnel and its 
output is processed through the DAU. Two "working standards" provide the test 
points. One is near vacuum and the other is the current atmospheric pressure 
which is actually sensed by another Ruska. In this experiment the "bits" output 
from the DAU are first processed using the calibration data developed for the 
appropriate channel. This information along with the values from the working 
standards is then used to determine the bias for a linear representation of the 
relationship between output voltage from the Ruska and pressure in engineering 
units. As one can see these experimental results will carry along the errors 
associated with the working standards, the DAU operation, the DAU calibration 
information and the curve fitting process. 

The final experiment is the actual in-tunnel, test point measurement as 
presented in Figure 10. In this case the voltage output from the Ruska, digitized 
using the DAU system, is converted to voltage using the DAU calibration data 
and then converted to pressure using the earlier developed pressure system 
calibration data. These pressures may be considered as results or, as in most 
cases, they are used with other information as part of the data processing to 
achieve the information desired from the test. These measured pressures have 
associated with them precision errors which could be quantified by making 
multiple measurements at a "fixed" test point as well as bias errors which are due 
to errors in standards, calibration constants, installation errors, and other factors. 

3.2.3 Temperature Measurement System 

There are two experiments performed with the temperature measurement 
system, as illustrated in Figures 1 1 and 12. One is a calibration performed in the 
calibration laboratory and the other is the actual temperature measurement 
performed in the tunnel at the test point. The PRT calibrations use constant 
temperature baths for the working standards. Though only limited details are 
indicated for the calibration laboratory process the results of the calibration are 
the Callender-Van Dusen constants, a, |3, and X which are then stored for future 
use. The working standard "baths" are monitored using another PRT sensor. 
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The in-tunnel total temperature measurement actually involves measuring 
two voltages. One voltage is associated with the PRT sensor located in the 
tunnel and the other is the voltage across a "precision calibration" resistance. 

Both voltages are digitized using independent DAU channels, and then the "bits" 
measured by the DAU are converted to digital "voltages" using the DAU 
calibration constants. The digital voltages, the value of the calibration resistor and 
the Callender-Van Dusen constants are then used in the data reduction program 
to determine the total temperature. As indicated earlier in the discussion on the 
pressure measurements, the error in total temperature will have both bias and 
precision components. The bias errors will be due to the calibration and 
installation of the PRT as well as bias associated with the DAU channel 
calibrations. 

3.2.4 Model Orientation Measurement System 

Though there are various means for determining the angle of attack of a 
wind tunnel model in the NTF, the system discussed in this report uses a single 
axis, servo-accelerometer located on the wind tunnel model to sense the relative 
orientation between the accelerometer and the local gravity vector. This sensor is 
referred to as an "AOA" in this report. The information acquired from this sensor 
is processed along with other information concerning flow direction and position 
of the sensor in the model in order to measure the angle between the free-stream 
flow direction and an appropriate reference axis on the model. 

In this case there are again three experiments associated with the 
orientation measurement, as illustrated in Figures 13-15. In the calibration 
laboratory, the working standard is provided by a precision indexing head which 
can be used to orient the AOA package to a variety of positions relative to the 
local gravity vector. Output from the instrument is digitized using an integrating 
digital voltmeter and the voltage and orientation information is "curve fit" in order 
to determine four parameters. These are the basic sensitivity, bias and two other 
parameters referred to as Omax and Az which are related to "misalignment" of 
the instrument relative to its mounting. Only the value for the sensitivity as 
determined in the calibration lab is actually used during data reduction and the 
other parameters are for reference purposes only. 

Once the AOA package is mounted in the model and the model mounted 
in the wind tunnel, an additional calibration experiment is performed, see Figure 
14. Using the model positioning system in the wind tunnel, the model is placed in 
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three different roll orientations, 0°, 90° and 180°. Using output from the sensor at 
these three measurement orientations and the instrument sensitivity from the 
calibration lab experiment, new values for bias, Omax and Az are determined. 
These values replace the reference values from the calibration laboratory 
experiment. (This may imply that the sensitivity and bias for the AOA are not 
independent parameters, since one is determined using the other. This could 
imply that bias errors in each are not uncorrelated - though this fact is not 
pursued in the current study. See Appendix B.) 

The in-tunnel measurement system for the model orientation at the test 
point is shown in Figure 15. The output voltage from the IDV is combined with the 
calibration information from the two calibration experiments to provide the 
orientation of the sensor relative to the gravity vector. This information is then 
combined with other information on the tunnel and model to determine the angle 
of attack as part of the data reduction process. 

As mentioned above the voltage output from the AOA is digitized using an 
integrating digital voltmeters for the calibration experiments and the tests 
conducted in the NTF. This is the only instrument of those considered in this 
report that does not use the DAU at the NTF for analog to digital conversion. As 
suggested by the dashed line in Figure 4, that capability does exist and has been 
used on some tests. It is obvious that one should be aware of the details of each 
aspect of the complete measurement system in order to be able to effectively 
assess the uncertainty. Changes in how the data is acquired or specified 
components of the system will influence the uncertainty estimates for the results. 

3.2.5 Force Measurement System 

As one might expect the balance is central to the aerodynamic force 
measurement system. The force balances are precision instruments, which in the 
case of the NTF, are exposed to an extremely hostile environment. The balances 
are approximately 2 inches in diameter and 15 inches long. They must support 
wind tunnel models that weigh hundreds of pounds and are subjected to 
aerodynamic loads that may be two orders of magnitude greater than the model 
weight. 

The calibration of the multi-component wind tunnel balance is in its own 
right a very complex experiment. The procedure is illustrated in Figure 16 and 
discussed in some detail in Reference 29. Loads are applied to the balance 
through a complex mechanical system using precision weights. The balances 
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can be calibrated at room temperatures ( as is done for most of the calibration 
information) or at cryogenic temperatures. During calibration the voltage applied 
to the balance bridges is measured as well as the voltage output from each of the 
six bridges. (The X6 symbol on the Figure is an indication that there are six strain 
gage bridge channels on each balance.) This is done for a wide variety of loading 
conditions and that data is process to provide a set of information referred to as 
the "balance deck". This includes the primary balance sensitivities, first and 
second order interaction coefficients, sensitivity shifts and zero shifts. As with any 
experiment each of these parameters which are the result of the calibration 
experiment has their own uncertainty. There is an ongoing effort to determine the 
uncertainty of each of these parameters and due to the complexity of this issue it 

is not discussed in detail in this report. 

The balance is then installed in the model and another calibration is 
conducted in the Model Preparation Area (MPA) at the NTF. This calibration 
takes place in a cryogenic chamber and uses the DAU system for analog to 
digital conversion. There are nine channels of balance information which are 
required, the six strain gages bridges and the voltage from three thermocouples 
mounted on the balance. The balance output is effected by the temperature of 
the balance and temperature gradients within the balance. This information is 
used to provide updated zero shifts for the balance as installed in the model. 

The balance measurement system in the tunnel is illustrated in Figure 18. 
The measured parameters consist of the voltage source, the six strain gage 
bridge imbalance voltages and the three thermocouple voltages. This data is 
combined with the calibration information to determine the forces transmitted 
from the model, through the balance and to the model support system. The NTF 
balance beams are configured to measure two normal forces, two side forces, an 
axial force and a rolling moment. These results are then converted into the three 
force components (axial, normal and side) and three moment components (pitch, 

yaw and roll) in the body axis system. 

It should be noted that though the data acquisition system used in the 
MPA is the same as that used in the tunnel that there are important differences in 
the systems. There are different "wires" which carry the very low voltage bridge 
output signals from the MPA to the patch panel, and thus the DAU, than from 
those that carry the signal from the tunnel to the patch panel. Though there are 
probably many other subtle issues not mentioned in this brief overview, it is this 
type of difference that can become important in an uncertainty analysis. A very 
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thorough understanding of the measurement system is required in order to 
perform an effective uncertainty analysis. This is particularly the case when the 
system being evaluated is as complex as the NTF. Most of the references cited 
earlier in the report suggest the use of caution concerning the calibration of 
sensors and instruments in environments outside of the nominal test 
environment. But as one can see, this is done in almost every measurement 
system used in the NTF and this must be accounted for in the uncertainty 
analysis. 

3.3 Basic Measurement Summary 

The previous section outlined the certain aspects of the various calibration 
and in-tunnel experiments. The following is a brief outline which provides an 
overview of a "typical" sequence of tests and their associated results. Each of 
these tests are required to provide the information needed to determine the drag 
coefficient at a single test point. For the calibration experiments ( a - k) a 
"standard" is indicated as well as the quantity/s measured and the result of each 
experiment. For the actual "measurements" conducted at a test point (I - p), the 
information recorded is indicated. This information is then combined with other 
data, often from the calibration experiments, to compute basic measurements 
(i.e. pressures, temperatures, forces, etc.) which in turn are used to determine 
results (i.e. Mach number, Reynolds number, angle of attack and Cp)- Defining 
the measurement process and sequence in this manner proved useful in 
attempting to establish the source of bias errors for the information used in the 
final off-line data reduction process. 

a. Ruska Calibrations 
standard - dead weight source 
measurement - voltages (IDV) 
result - "slope" constant 

b. PRT Probe Calibration 

standard - constant temperature baths 
measurement - voltages (source unknown) 
result - Callender-Van Dusen constants 
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c. Single axis AOA Calibration 

standard - mechanical positioning system 
measurement - voltages (IDV) 
result - "sensitivity" constant 

d. 6-component Balance Calibration - Ambient temperature 

standard - precision weights and mechanism 
measurement - voltages (IDV) 
result - "Balance deck" 

e. 6-component Balance Calibration - Cryogenic temperatures 

standard - precision weights and mechanism 

measurement - voltages (IDV) 

result - sensitivity shift correction parameters 

f. 6-component Balance/Model - MPA- Cryogenic temperatures 

standard - MPA temperature sensors 

measurement - binary numbers, 0 to ± 2^ 4 for nine channels 
result - zero shift temperature correction parameters 

g. Data Acquisition Unit Calibration - per channel 

standard - precision voltage source 
measurement - binary numbers, 0 to + 2^ 4 
result - slope and bias constants for each channel 

h. Ruska Two-point Calibration 

standard - vacuum and Ruska 
measurement - binary numbers, 0 to ± 2 14 
result - zero shift voltage 

i. Single axis AOA In-tunnel Calibration - ~T ^ m b j ©pit 

standard - reference block and tunnel model positioning system 
measurement - binary numbers, voltage from IDV 
result - bias and misalignment corrections 

j. Model Weight Tare Measurements- T ambjent 

standard - model positioning system 

recorded data -binary numbers, 0 to ± 2 14 , balance output 
result - weight tare corrections 
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k. Wind-off Zero Measurements - Cryogenic test point temperature 

standard - model positioned at reference condition 
measurement - binary numbers, balance output 
result - wind off zero shift voltages 

l. Ruska Total Pressure Measurement - Test point conditions 

recorded data -sensor output voltage converted to binary number 
0to±2 14 

m. Ruska Static Pressure Measurement - Test point conditions 
recorded data -sensor output voltage converted to binary number, 

0 to ± 2 14 

n. PRT Probe Measurement - Test point conditions 

recorded data - sensor output voltage converted to binary number 
0to±2 14 

o. Model Attitude Measurement - Test point conditions 

recorded data - digital representation of sensor output in volts from IDV 

p. 6-Component Balance Measurement - Test point conditions 

recorded data - balance output voltage converted to binary numbers, 0 to 
± 2 14 , for nine channels 

3.4 Off-line Data Reduction 

The NTF data system allows for both on-line and off-line data processing. 
The on-line systems provides near-real-time process parameters and "results". 
Not all corrections are applied to the on-line results and therefore they are often 
considered to be "approximate". This on-line capability provides "snapshots" of 
time-averaged information and provides the test directors and test engineers with 
information used to make decisions during the testing in the form of listings and 
data plots. 

A majority of the data processing is conducted after the completion of the 
test and allows for more detailed analysis and re-analysis of the recorded 
information. This report is concerned with the uncertainty in the results 
developed using the off-line data processing. The primary "components" of the 
data reduction system are the computer and primary data processing software, 
the "DAS" tape and the setup "deck". 

The basic software is developed from FORTRAN code and can be 
executed on a variety of computers. It is a rather extensive computer program but 
allows the user significant flexibility and numerous options for data processing. 
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The program also contains certain specific information such as "gas" constants 
and tunnel calibration information which cannot be altered by the user. The 
"DAS" tape is the primary source of recorded information and data. It contains 
the digital representation of the analog measurements from the DAU, the DAU 
calibration results and Ruska calibration constants. Most of this information is 
considered to be "raw" and the data has undergone very little processing. 

The "setup" deck provides the user with a direct interface with the data 
reduction process. It contains the force balance, AOA and PRT calibration 
constants. It identifies the appropriate DAU output channels for data reduction 
and any special instructions or calculations requested by the user. When the 
study documented in this report was conducted, alteration of the "setup" deck 
was the only means available to evaluate the data reduction process. Therefore 
the parameters which were considered in the sensitivity analysis and subsequent 
error calculations were limited to those which could be accessed through the 
"setup" deck. This was due to the limited time available for the current effort and 
not an inherent characteristic of the data reduction process. 

There were five parameters which were considered as the "results" in this 
preliminary study. These were, 

Mach Number 
Reynolds Number 
Angle of Attack 
Drag Coefficient, Cd 
L ift Coefficient, Cl 

These represent only a very limited subset of the literally hundreds of process or 
model parameters which can be computed as part of the data reduction process. 
Each are determined by a number of constants and measured data. Additional 
details on the methods used for data reduction carnbe found in Reference 29. 

The uncertainty in each of these results will be influenced by the methods 
used to calculate the results. Certain issues such as computer precision and 
approximate numerical methods or approximate analytic models can influence 
the uncertainty estimates. These issues have not been addressed in this study 
and though it is not apparent that they will have a significant influence on the 
uncertainty estimates for the NTF, their effect should be considered as this 
uncertainty analysis moves from its preliminary to the application stage. 

As related to data reduction, the most critical aspect which can influence 
the uncertainty in the result is a sound understanding of the "physics" which 
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influence the result. If improper models or assumptions are applied to the 
"measured" results, the "errors" can be significant even if all of the computations 
and numerical calculations are "exact". An inappropriate equation of state, 
erroneous gas constant, or an assumption that the pressure at point "a" is equal 
to that a point "b" can result in an uncertainty which exceeds all of those 
estimated for the actual measurement process. As is the case with the 
measurement process, a thorough understanding of the "physics" is a 
prerequisite for an effective uncertainty analysis. 
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Section 4. Calibration and Uncertainty 


Computer-based, automated data acquisition and processing systems 
have provided significant improvements in the manner in which experiments are 
performed but they have also provided some additional challenges. They can 
allow for the acquisition of large amounts of information and on occasion may 
encourage the user to consider a result to be "correct" because the computer 
processed the data with great precision. In determining the result of an 
experiment, the computer usually manipulates two types of information; data 
acquired as part of the actual measurement and "constants", some of which are 
the results of other experiments often referred to as calibrations. This section 
addresses a number of issues related to the data developed in calibration 
experiments and subsequently used in the primary measurement process. 

Calibration is the method used to exchange "large" bias errors in an 
instrument for the "smaller" bias errors of the standard used for the calibration 
and the precision errors in the calibration process. In the case of the NTF, all of 
the instruments used in the wind tunnel tests are calibrated, some on a daily 
basis. Most of the publications related to uncertainty analysis strongly 
recommend that instrument calibrations be performed under the same conditions 
as the actual measurement process. As indicated in Section 3, in the case of the 
NTF some of the calibrations are conducted in the tunnel itself but often the 
calibration is performed in a specialized calibration laboratory. Since these 
calibration experiments provide a majority of the "data" used in the calculation of 
the results of a wind tunnel test, the information developed during calibration and 
its uncertainty are an important contributor to the overall uncertainty of the result. 

4.1 The Calibration Experiment 

Quantifying the influence of the uncertainty in the "instrument" as it will be 
used in the actual measurement process is complicated when the calibration of 
the instrument is conducted in an environment and with data acquisition systems 
which are different than those used with the instrument in the wind tunnel. When 
this occurs one may wish to consider that the calibration process is actually a 
separate "experiment" from which specific information is determined and used in 
the subsequent application of the instrument. In this case it is important to realize 
what information is actually provided by the calibration process and to establish 
the required "uncertainty" for that information. 
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There are a number of assumptions upon which the approach for dealing 
with calibration uncertainty as presented in this report are based. They are: 

1 . "Calibration ” is simply an experiment whose results are specific "numbers" 
which will be used in other experiments which are referred to as 
measurements. Uncertainty associated with the results of the calibration 
are "fixed" or "frozen" when used in the subsequent calculations and 
contribute to the bias errors in the actual measurement. 

2. Automated, computer-based data acquisitions systems are used to 

measure" voltages and only voltages. Actually the data acquisition 
systems produce finite precision, binary numbers which are hopefully 
proportional to voltage over a specified range. The voltage is the result of 
data reduction using the binary representation of the input signal and other 
"calibration" information. 

3. The signal conditioning and analog-to-digital (A/D) systems used in the 

"calibration" experiment and the "measurement" system are/may not the 
same systems. 

4. The "environment" in which the calibration takes place may not be the 

same as that for the measurement. 

Consider the simple schematic shown in Figure 19. It represents two 
distinctly different experiments although some of the hardware is common to 
both. In the first experiment, referred to as "calibration", some type of 
sensor/transducer is exposed to an environment which results in a response from 
the transducer. This response is typically processed by a collection of electronic 
devices which make-up the "instrument". The instrument provides a voltage 
differential which is in some way related to the response of the sensor to the 
input provided by the "calibration" environment. This voltage is then introduced 
as input to a signal conditioning circuit, SP1 , where it may be filtered or amplified 
or both. This modified voltage difference is then provided to an analog-to-digital 
(A/DI) converter which generates a "number of bits". This "number of bits" 
information, now in the form of digital information, is usually converted to a 
number with a finite number of significant digits which represents the voltage 
resulting from the sensor's response to the environment . It should be 
emphasized that each element in this process can be influenced by its 
environment and operating conditions and each of these influences can add to 
the uncertainty in the information developed. 
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In a parallel fashion, additional information is provided by a "working 
standard" and the results are digital information which contain a finite number of 
significant figures. It is expressed in the units of the physical quantity being 
measured, often a physical quantity expressed in engineering units, EU. 
Therefore the basic information recorded during the calibration experiment are 
digital data pairs (volts, EU) and each of these quantities has an associated 
"uncertainty" or "error". The EU error is due to the uncertainty in the working 
standard and the voltage errors are due to the processing and recording of the 
instrument output. One should note that for some instruments the data pairs are 
actually pairs of vectors of information. This is the case for the six-component 
wind tunnel balances used in the NTF. Though much of the following discussion 
is conceptually consistent with this more complex instrument, many of the 
practical issues are far beyond the scope of this report. 

One needs to determine the uncertainty in the "results" of the calibration 
experiment which are typically calibration "constants" which are then used to 
process the information provided by the instrument in the wind tunnel. Before 
considering how to estimate the uncertainty in the calibration experiment, again 
refer to Figure 19 and the actual "measurement" experiment. In this case the 
same instrument is now used but in a different manner. The transducer/sensor is 
exposed to the test environment and again the instrument produces a voltage 
differential. This voltage differential is applied to another (most often with different 
characteristics) signal conditioning system, SP2, and a different A/D system 
DAU2. The "bits" information from DAU2 are converted to "voltage" in digital 
form. That voltage has a certain "error" associated with it and it undergoes 
numerical processing (data reduction) in order to provide a measure of the 
physical parameter in engineering units. The numerical processing involves the 
calibration constants determined from the calibration experiment. Therefore the 
measurement, expressed in engineering units, carries with it uncertainty due to 
both the calibration and measurement process. 

4.2 Curve Fitting 

The calibration constants that are used to characterize a given 
instrument's performance are often determined from a linear regression analysis 
or "least-squares curve fit". Curve fitting is a data processing procedure in which 
the basic concepts of propagation of errors or statistical analysis can be applied 
in order to estimate the required uncertainty information. References 22 and 24 
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present detailed developments and associated numerical procedures for 
regression analysis so the following only highlights aspects that are directly 
related to the methods applied in this study. 

Consider the results of the calibration experiment to be ordered pairs of 
data. Recall that each piece of digital information has associated with it an error. 
Most developments for least squares curve fitting are based upon the 
assumptions that there is no uncertainty in the independent variable and that the 
dependent variable satisfies certain statistical requirements. This assumption is 
quite reasonable in cases where the calibration process is "steady" and the 
uncertainty in the working standard is significantly less than the precision of the 
instrument. This assumption may not be correct in cases where the precision of 
the instrument and the working standard are comparable. 

Assuming that these assumptions are satisfied then the sets of data are in 
the form, (Xj , Yj) for i = 1 ,N. For the simple case considered in this report the 
least squares curve fit will be used to quantify a linear relationship between X and 
Y and provide the slope, m, and intercept, a, for the expression: 

Y = mX + a Eq. 4.2.1 

For the straight-line fit a rather simple expression can be written for the two 
"results" a and m explicitly in terms of the "measurements" Xj and Yj for all N 
measurements. 

N N N 

N £Xj Yj-X Xj X Yj 


i=1 i=1 i=1 



a = 




N 

( Xj Yj ) X Xj 
i=1 



Eq. 4.2.3 
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Since the parameters used to determine a and m had associated with them an 
uncertainty (at least the Yj's did), then both a and m also have an uncertainty. 
Estimating this uncertainty is an important part of the regression analysis since it 
provides an indication of the utility of the resulting regression coefficients. 

A number of approaches can be used to estimate the uncertainty in these 
parameters but considering the context of this report, an approach based upon 
error propagation was used. If one considers the determination of a and m as 
simply a data reduction process using numbers with their own uncertainties, then 
the uncertainty in a and m can be determined using the methods outlined in 
Section 2.7. This requires an estimate of the uncertainty in each measured 
parameter. If one assumes no uncertainty in the independent variable, then the 
precision index for x, which is S xi , is zero. The precision index for the dependent 
variable can be based upon the unbiased estimate of the standard deviation 
associated with the dependent variable can be expressed as, 



Eq. 4.2.4 


This parameter is also referred to as the standard error of the estimate (SEE). 17 
This estimate is then used with Eq. 2.7.2 as applied to the computed parameters 
"a and m" to yield estimates of the precision indices for a and m, 
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Eq. 4.2.5 
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Eq. 4.2.6 


The precision limit can be determined from the precision index by scaling with the 
appropriate t factor for the desired level of confidence and appropriate degrees of 
freedom. The degrees of freedom for the estimate of the standard deviation is 
based upon the number of points in the "fit" and the number of coefficients. In this 
case it is N-2 where N is the number of points to which the straight line is fit. 

Once the two estimates of the precision limits for the parameters a and m 
have been determined they become fixed as bias errors and are treated as such 
in the remaining calculations. 

B a = tS a Eq. 4.2.7 

B m = tS m Eq. 4.2.8 

This is due to the fact that these parameters are "frozen" until the next time a 
curve fit is conducted. 

4.3 Bias Errors in the Measurement Due to Calibration 

In the case of a calibration experiment, the independent variable, X, is 
usually the value of the working standard and is often expressed as a quantity in 
engineering units. The dependent variable, Y, is the sensor output often 
expressed as volts. When the calibration experiment is complete, the curve fit 
performed and the uncertainties computed, the instrument is now used in a 
measurement. When this occurs the expression, 

Y= a + m X Eq. 4.3.1 

is then rearranged and "used" in the form, 

XM = A + M Y Eq. 4.3.2 
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where XM is the measured parameter in engineering units corresponding to the 
voltage Y. The slope and intercept for this reciprocal relation are, 


A = - 


a_ 

m 


Eq. 4.3.3 


M = 


1 

m 


Eq. 4.3.4 


Since A and M are the result of computations using 'a' and 'm', one can estimate 
the bias limits for A and M from the bias limits for 'a' and 'm' using propagation of 
errors, Eq. 2.7.2, and assuming that the bias limits in 'a' and 'm' are uncorrelated. 
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Evaluating these two expressions yields 
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Then in the process of using the calibration constants to evaluate the "measured" 
parameter XM the bias due to the curve fit may be combined with other bias 
limits. These other bias limits can be attributed to the working standard or other 
sources as the particular application requires. Then the bias limits in A and M are 
computed by addition in quadrature. 
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Eq. 4.3.10 


This form is based upon the assumption that the biases in the curve-fitting and 
calibration process are uncorrelated. The bias limit for the "measured" parameter 
XM is then computed recalling Eq. 4.3.2 and Eq. 2.7.2 where the bias error in 
the two computed parameters A and M as well as the bias in the measured 
parameter Y are combined and propagated through the data reduction process to 
determine the bias in XM. 
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Which results in the expression, 
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This bias limit is expressed in engineering units in the measured result. It could 
be combined with other bias limits in XM due to estimated biases in the 
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measurement process or installation. If the parameter XM is a result, the bias 
limit could be combined with a measured precision limit in parameter XM 
(determined from multiple measurements) to yield an uncertainty in XM. Or as is 
the case for most of the measurements, it can be propagated into the calculation 
of other results to determine the bias limits of the computed results. 
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Section 5. Uncertainty Estimation - Preliminary Results 


This section presents the results of a preliminary end-to-end system 
uncertainty analysis performed using information from the NTF. It should be 
emphasized from the beginning that this represents a "framework" for a more 
comprehensive uncertainty analysis and the "approach" associated with each 
step is of much greater concern than the "results." As with any uncertainty 
analysis, as the results of this preliminary study are evaluated, additional factors 
can be introduced and parameter estimates, particularly with regard to the bias 
estimates, can be improved. There are many results which are determined from a 
test in the NTF. In this study the primary results for which uncertainty estimates 
were made were the Mach and Reynolds numbers, angle of attack and the non- 
dimensional force coefficients in the wind-axis system Cl and Cq. 

5.1 Nominal Test Point 

Though one may be tempted to refer to a single "uncertainty" for an entire 
experiment or facility, in practice, each data point or measurement has 
associated with it its own unique uncertainty estimate. This is particularly 
important in those cases where the parameter sensitivities depend upon the 
nominal parameter values - a most common occurrence. For the study presented 
in this report, data for a series of tests on a subsonic commercial transport model 
were available. The basic test conditions are summarized as, 

Mach No. = 0.80 
Reynolds No. = 40.7 x 10^ 

Total Temperature = -250° F 
Dynamic pressure = 2660 psia 

This represents a near full scale test simulation which is the purpose of the 
cryogenic capabilities of the NTF. The tests were conducted in nitrogen in order 
to achieve the cryogenic conditions and thus proper Reynolds number scaling. 

The wind tunnel balance NTF1 13A was used for these tests. The "angle of 
attack" or AOA device was designated as 12-6. Each of the balances and 
orientation sensors used at the NTF have their own designation, calibration and 
calibration history. The detailed information available for each of these devices 
was invaluable in preparing the uncertainty estimates. 

Most of the results presented in this study are for the two test points 
shown below. These represent only two test points in a large series of tests 
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conducted with this model. These test points were selected so that the same 
DAU calibration and wind-off-zero information was used for each test point. The 
two points represent a near "zero-lift" case and a near cruise case and will be 
referred to as Test Points A and B in this report. 


Test Point 

Angle of Attack 

C D 

c L 

A 

-0.92° 

0.01517 

0.034 

B 

2.18 

0.02144 

0.394 


These two test points represent "single" measurements and therefore precision 
error estimates conducted using either would be based on single point 
measurement techniques. During the test on this particular aircraft model a series 
of measurements were also conducted which allowed for the use of multiple 
measurement techniques for the precision limit estimates. One of these tests was 
considered in some detail in the current study. In this case 10 sequential 
measurements were recorded. The model was positioned at the desired angle of 
attack and the tunnel conditions established. The computer was manually 
signaled to record a data point, the conditions were maintained and then after a 
delay of about 1 0 seconds another data point recorded. This process was 
repeated until the 10 data points had been measured. The average values for the 
model conditions are given below and the other test parameters were the same 
as those for test points A and B. These 1 0 measurements will be referred to as 
Test Points C through L in this report. 

Test Point Angle of Attack C D c |_ 

C_|_ 1.64° 0.01935 0.357 

These 10 test points were then considered in more detail as discussed in the 
following sections and were used to establish end-to-end precision limit 
estimates. 

5.2 Sensitivity Calculations 

In order to determine the uncertainty in the experimental results one must 
determine the "sensitivities" of the results to variations in the contributing 
measurements as discussed in Section 2.7. There are two options for 
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determining these derivatives. The first is by analytic evaluation of the partial 
derivatives as was done in Reference 23. Though in certain cases this is both 
very direct and provides additional useful information, it was not the approach 
taken during this study. Since the actual data reduction software which is used to 
process the measurements taken in the NTF was available, the sensitivities were 
determined from numerical estimates of the partial derivatives. This involved a 
systematic perturbation of the individual parameters about the nominal test 
conditions. 

Due to the time constraints imposed upon this study the only parameters 
which were included in the uncertainty estimate were those which sensitivities 
could be determined directly from the data reduction software. This limited the 
analysis to include only those parameters which could be altered via the SETUP 
data for the off-line data processing. Certain measurements and constants which 
are stored on the DAS tape were not considered but it must be emphasized that 
this was not necessarily due to the fact that they were not important, only that 
they were "inaccessible" during this preliminary study. 

Two types of parameters were considered during the sensitivity 
calculations. They were classified as "fixed" parameters and "measured" 
parameters. The fixed parameters were those developed during previous 
calibration experiments or tests. The measured parameters were unique to a 
given test point and are the average values of the 10 analog to digital 
conversions at a 10 Hz sample rate. The data processing program automatically 
applies the DAU calibration information to the "bit" information from the analog to 
digital converter so the that the measured parameters are in units of voltage. 

The following list includes all of the parameters considered during this 
preliminary study. The notation used to identify each parameter is consistent with 
the data processing program and associated documentation 27 . 

1 . Balance sensitivities: SI - S6 [fixed] 

(note the primary sensitivities correspond to the following force 
components : SI - AF, S2 - SF, S3 - NF, S4 - RM, S5 - PM, S6 - YM ) 

2. Balance input voltage: BALPOW [measured] 

3. Balance bridge output voltages: NF1RAW, NF2RAW, SF1RAW, SF2RAW, 
AXFRAW, RLMRAW [measured] 

4. AOA output voltage: ELEVA [measured] 

5. AOA calibration constants: AOAS, AOAB, AOAZ, AOAM [fixed] 
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6. Weight Tare results: WNF1 , WAF1 [fixed] 

7. Ruska output voltages: PT100V, PS50V [measured] 

8. PRT calibration constants: RO, alpha, beta, lambda [fixed] 

The primary balance sensitivities were incremented by fixed values which ranged 
from approximately 0.1% to 1% of their magnitude. The remaining parameters 
were incremented by 1% of their magnitude. No consideration was taken at this 
time as to the accuracy of the estimate for the gradient. The nominal value of the 
result with the unperturbed parameter and the value resulting from the perturbed 
parameter were used with Eq. 2.7.4 to estimate the sensitivity and then Eq. 2.7.7 
was used to determine the relative sensitivities for each test point. 

The relative sensitivities were computed using a simple computer 
spreadsheet program and the results for test point A and B are given in Tables 1 
and 2. The relative sensitivities were computed for the primary results which were 
considered as part of this study: Mach and Reynolds numbers, angle of attack, 

Cl and CD- In the various columns in Table the symbol "P" refers to the 
individual parameter, thus for example, dRe/Re/dP/P in the row associated with 
the parameter PT100V, is the relative sensitivity in Reynolds number to variations 
in the output from the Ruska used to measure total pressure. Though these 
calculations were conducted in a very tedious, manual fashion for this study, one 
could consider the automation of this process through modification of the existing 
data process code or possibly with recent developments in automated 
differentiation software. 

5.3 Bias Error Estimates 

As indicated in the earlier discussion the bias error estimates are only as good as 
the "estimator's" judgment and experience. This is particularly true for a system as 
complex as the NTF where there are many different potential sources for bias errors. 
The approach used in this study was to make a "reasonable" but conservative 
estimate of the bias error. Then the contribution to the overall error in the result 
(Cd, Fie, M, etc.) was determined. If the effect was an insignificant contribution 
no additional effort was made to improve the estimate. If the effect was 
significant, then consistent with the time and resources available, a somewhat 
less conservative estimate was established. 

The first set of parameters considered were the fixed parameters. Since in each 
case these were the results of earlier calibration experiments, the experimental 
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uncertainty in the result from the calibration was "frozen" as a bias error in the actual 
measurement. The following briefly outlines the information source for the bias error 
estimates for the fixed parameters. Appendix C details the calculations for each. 

1 . Balance Primary Sensitivities 

There were two sources of information available for estimating the bias 
errors in these parameters. A number of the NTF balances have 
undergone repeated calibrations therefore there is a history of calibration 
results and two calibrations conducted within the past two years were 
available for balance NTF1 13A. These were used to make a very crude 
estimate of the precision index for the calibration process. Along with the 
calibration parameters there is also an estimate of the "accuracy" of each 
force or moment component as measure in the calibration experiment 31 . It 
was assumed that these percentage errors could be used as estimates of 
the percentage bias limits in the primary sensitivities. At the time this 
research was being conducted, the Instrumentation Research Division at 
the NASA Langley Research Center was working to develop techniques 
based upon statistical assessment of the large amount of data recorded 
during the calibration experiments to provided improved estimates of the 
uncertainty in all of the balance calibration constants. One would 
anticipate that this work will result in improved estimates for the bias errors 
in all of the fixed parameters associated with the balance. 

2. AOA Constants 

The primary sensitivity is determined in a calibration laboratory 
experiment. Only two calibrations were available for the 12-6 AOA device 
but 5 were available for the comparable 12-4 instrument. An in-tunnel 
calibration experiment was performed to determine the three other AOA 
parameters, the bias, Az, Omax. Fortunately, data from three separate 
calibration experiments for this particular test were available and simple 
statistics were used to estimate the precision indices associates with these 
results. The resulting precision limits from the calibration experiments 
were then frozen as bias errors for the subsequent measurements. 

3. PRT Constants 

The bias estimates for these constants were developed in the same 
manner as those for the AOA sensitivities. Three calibration results for a 
"typical" PRT system were available. An estimate of the precision limit due 
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to the calibration process was frozen as the bias limit for the instrument 
calibration constants. 

4. Tare Weight 

Since little data was available for this parameter, a simple estimate based 
upon the difference in the model weight estimate from the axial and 
normal force measurements in the tare weight test was used as the bias 
limit. This bias limit estimate was then applied to both parameters WNF 1 
and WAF1 . 

It must be emphasized that the calculations shown in Appendix C represent 
preliminary estimates for these bias limits and that continued evaluation of the results of 
the calibration experiments which determine these fixed parameters are required in 

order to provide more realistic estimates. 

The bias errors in the measured parameters are due to uncertainties in the data 
acquisition process which include instrument installation , the analog-to-digital data 
conversion process, and data processing. For this preliminary estimate only a limited 
number of contribution factors were considered. These are briefly outlined below. 

1 . Analog input to DAU 

- Uncertainty in working standard - manufacturer 

- Data reduction, calibration curve fit (bits vs. volts) 

- Data processing (bits to volts) 

2. Other analog to digital conversion processes, IDV 

- Manufactures specifications 

As one evaluates these bias estimates it is important to recognize that factors related to 

the instrument installation have not been considered. 

Appendix D contains the sample calculations used for one channel of the DAU. 
For this particular experiment it was Channel 18 which was used to provide the input 
voltage from the axial beam of the balance. Similar calculations were performed for 
each of the measured parameters. 

Tables 3 and 4 summarize the bias error estimates for Test Points A and B. The 
Tables include the value of the parameter and its units. In the case of measured 
parameters the DAU channel and its range are also indicated. The various bias factors 
are added in quadrature (Eq. 2.6.1) and the bias error estimate listed. The Tables also 
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indicate the non-dimensional relative bias in percent of the nominal value of the 
parameter for that Test Point. 

Since some of the bias estimates depend upon the value of the nominal value of 
the parameter, they differ between test points. Others are the same for each test point. 
The largest relative bias errors exist for the AOA parameters determined as part of the 
in-tunnel calibration and the "beta" coefficient in the Callender-Van Dusen equation 
used for the PRT probe. 

5.4 Propagation of Bias Errors 

Once the bias limit estimates and sensitivities have been developed for the 
appropriate fixed and measured parameters, it is possible to estimate the bias limit in 
the results. The biases in the results were computed using a computer based 
spreadsheet program and Eq. 2.7.5. The size of the spreadsheet preludes its 
presentation in this report and the results presented below are summarized from 
calculations performed using the spreadsheet. Due to the repetitive nature of the 
calculations and the "dynamic" nature of the uncertainty estimates, the spreadsheet 
appears to be an ideal vehicle for off-line, post processing of these results. If the 
sensitivities were computed as part of the data reduction process, it is feasible that the 
bias error estimates could be made as part of the actual data reduction software. This 
approach would require a significant modification to the current NTF software and was 
not considered as part of the current study. 

The assumption was made that all of the parameters used in this study were 
independent and thus the problem of correlated bias errors was eliminated. In the case 
of the measured parameters this assumption appears quite valid since each 
measurement is performed with its own instrument. There may be reason to further 
examine this assumption for a number of the fixed parameters. These parameters have 
been determined based upon measurements performed in earlier calibration 
experiments and as illustrated in Appendix B, if the bias errors in these parameters are 
correlated they will influence the bias estimate in the computed result. This does seem 
somewhat unlikely based upon the bias estimates made in this report but as these 
estimates are refined, one should determine if the estimates are correlated. 

The individual terms which are added in quadrature in Eq. 2.7.5 are referred to 
as the "relative bias factors" in this report. Thus the relative bias factor in result "X" from 
parameter "i" is defined as, 

• B Pj 2 

X BF= (Qj “p j ) Eq. 5.4.1 
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By ranking these non-dimensional bias factors for each of the results computed as part 
of this study, one is able to establish the dominate error sources for each result. Tables 
5-9 illustrate the bias factors for the primary results Mach and Reynolds numbers, angle 
of attack, and the non-dimensional force coefficients Cl and Cq for Test Point A. Similar 
results for Test Point B are provided in Tables 10-14. In each Table only those 
parameters which have contributions within a couple of orders of magnitude of the 
dominate factor are included and any others, though they may be non-zero, were 
ignored. 

The bias limit estimates for each Test Point are summarized in Tables 15 and 16. 
The bias limit estimates are listed with a single significant figure (the 1 plus next digit for 
numbers starting with a 1) and the result is "rounded" to correspond with the magnitude 
of the bias error. The variation in error with test point is indicated for those results which 
vary between the test points as would be expected. 

5.5 Precision Error Estimates 

Unlike the case for the bias errors where both fixed and measured parameters 
were considered, the precision error estimates are only influenced by the measured 
parameters for these two test points. The fixed parameters do not vary between test 
points for the cases considered in this study. As indicated above the classification of 
"fixed" and "measured" parameters is influenced by the manner and sequence in which 
the tests are conducted. If AOA in-tunnel calibrations were repeated between the test 
points, then precision errors in the bias, Az and Omax parameters would add to the 
precision error estimates. With the data available for this study there were two 
approaches considered to estimate the precision limits in the results. In the first case a 
single result is computed based upon the mean value of multiple measurements of 
individual parameters. In the second case multiple results are computed ( each based 
upon multiple measurements of individual parameters ) and a statistical analysis of the 
multiple results provides an estimate of the precision limit in the result. 

5.5.1 Single Result with Multiple Measurements 

In Section 2.5 the concept of precision error was overviewed. At that point it was 
stated that the preferred approach to quantifying the precision error was to acquire 
multiple measurements of the same result and determine the precision index and thus 
the precision limit from those multiple results. In many wind tunnel applications this is 
possible though it can add to the time required for a test program and thus the cost. 
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When computer based data acquisition systems are used a single parameter 
measurement is often actually the average of multiple measurements sampled over a 
specified time increment so there is additional information available to help determine 
the contribution to uncertainty due to precision errors. For the case where a single 
result is computed and based upon the average of multiple individual measurements, an 
estimate of the precision index in the result can be determined by quantifying precision 
index of the individual parameter measurements. Then propagation of errors is used to 
estimate the precision index of the result and with the appropriate degree of freedom, 
one can determine the precision limit and associated coverage. Since this type of 
information was available for this preliminary study this approach was also used. 

For the test program which provided the data from the NTF for the current report 
a single test point was computed by sampling 10 samples of each measured parameter 
during a 1 second interval. The 10 samples were averaged and the result was 
computed based upon the mean value of the parameters. By evaluating the statistical 
variation of each of these parameters during the sampling interval and using the 

propagation of errors, one can estimate the precision error in the result. 

The precision index for "i th" parameter, P. { , can be determined using Eq. 2.3.4 to 
yield Sp. . The estimate of the precision index for the mean value of Pj is, 

S Pi 

Spi = -j= Eq. 5.5.1 

and since the mean value of Pj is used in the calculation of the result, the precision 
index for the mean is used to determine the precision index of the result. There are a 
number of approaches that can be used when dealing with the sample size and its 
influence on these statistical estimates. The AGARD Draft recommends the use of the 
large sample size approximation for any case, i.e. always assume a "t" factor of 2 and a 
95% coverage. Reference 16 provides a number of approaches for dealing with variable 
sample sizes and determining the degrees-of-freedom associated with the precision 
index of the result. Since for the data consider in this study the same number of 
samples were available for each parameter, the number of degrees-of-freedom 
associated with each estimate of the precision index are the same. This allows one to 
determine the precision limit for each parameter based upon the appropriate sample 
size. In this case, there were 10 points per sample, thus 9 degrees-of-freedom for each 
sample and a t factor of 2.26 for a coverage of 95%. The precision limit for the mean 
value of the "i th" parameter is written as, 
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Eq. 5.5.2 




The relative precision limit for the result is then determined from, 



i = 1 


where Pj is the mean value of the "i th" parameter. 

The precision limit of the mean for the measured parameters for Test Point 
A are given in Table 17. This information was then combined with the sensitivities 
for this test point and Eq. 5.5.3 was used to determine the estimate of the 
precision limit for the result. This was also performed using a computer 
spreadsheet program. As was done in the previous section a relative precision 
factor for result “X" was defined as, 


, PLpj 

XpF = (®j ) 


p i > 2 


P. 

i 


Eq. 5.5.4 


This allows for ranking of those factors which have the greatest influence on the 
precision limit of the result. Sample results are included in Tables 18 and 19 for Test 
Point A. For these two results at this test point the precision error is dominated by the 

fluctuations in the strain gage bridge output. 

The precision limits estimated from Eq. 5.5.3 are shown in Tables 20 and 21 for 
Test Points A and B. The same type of rounding and truncation as applied to the bias 
limit estimates were used with these results. A brief discussion of these results in the 
light of a more detailed assessment of the measurements is included in Section 5.5.3. 
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5.5.2 Results from More Than One Test 
As indicated above repeated test information was available in order to establish 
precision error estimates using multiple results and thus provide an end-to-end precision 
limit assessment. The precision index for a sample of N results computed using 
measurements taken at the "same" test conditions is, 



Eq. 5.5.5 


Results from a sequence of 1 0 test points which were collected in succession with 
approximately a 10 second interval between test points are given in Table 22. This 
Table also includes estimates of the precision index and precision limits (based upon 
95% coverage) for each result. A comparison of these estimates with those obtained 
using the procedure described in the previous section indicate that these estimates are 
higher in almost every case. One should also recall that the both Test Points A and B 
were at a different angle of attack than the sequence of Test Points C - L. These results 
did indicate that one should consider in more detail the information used to develop the 
two estimates. 

5.5.3 Qualitative Assessment of Precision Errors 

As emphasized earlier, this study was intended to develop a framework for a 
system level uncertainty analysis for a facility like the NTF. Even though it was not 
intended to provide a comprehensive uncertainty assessment of the facility, the results 
of the previous two sections indicated that a somewhat closer examination might 
provide some insight into the differences in the precision error estimates. 

Figures 20 - 22 present the 10 individual samples, measured in bits, for Test 
Point A for three of the measured parameters, AXFRAW, PTV100 AND ELEVA. The 
two parameters AXFRAW and PTV100 are output from the DAU and therefore the 
analog signal was passed through a 1 Hz low-pass filter before the conversion process. 
The ELEVA output was from the integrating digital voltmeter which has an output update 
rate of approximately 0.33 seconds. These figures imply that these relatively small 
samples are not normally distributed "random" numbers as was assumed in computing 
the precision index for each of these parameters. Considering the argument used in the 
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discussion for Figure 2, it appears that a 1 second sampling interval and a 10 Hz 
sample rate may not provide an adequate "sample" for the precision index estimates. 

Figure 23 presents the value of the result, C D> for Test Points C - L. Simple visual 

inspection of this Figure implies that this may be a more realistic "random" sample. One 
could perform a number of statistical tests on this sample to determine if it was 
representative of a normal distribution. An alternative presentation of this data is 
provided in Figure 24 which is a plot of C D as a function of angle of attack. From this 
Figure it appears as if these results are rather well correlated indicating a systematic not 
random variation in both. Even though these were intended to be identical test points 
at a fixed angle of attack, there are other factors which do influence the measurements. 
During the data collection, the process control system is continually monitoring the 
Mach number, total pressure, total temperature, and angle of attack. Continuous closed- 
loop control of each of these parameters produces systematic variations in each which 
may be reflected in the experimental results. Extracting short term samples of any of 
these parameters may not yield appropriate statistics to characterize the true random 
character of the measurements. 

Though one cannot draw any significant conclusions from this brief 
assessment of the measurements, it does imply that one should not rely on 
simple "statistics" in order to assess the precision errors. You must evaluate the 
results and the data that produced them. It also points out one of the useful 
aspects of performing an uncertainty assessment since the uncertainty analysis 
requires a careful evaluation of data sampling procedures and basic data 
characteristics. 

5.6 System Uncertainty Estimates 

The final step in the uncertainty analysis is the combination of precision and bias 
limit estimates to form the uncertainty estimate. Eq'. 2.7.10 was used to estimate the 
uncertainty in each of the results considered in this study. The addition in quadrature 
provides a 95% coverage or confidence level which is consistent with the assumptions 
made throughout this study. 

Tables 23 and 24 present the results of the uncertainty analysis for Test Points A 
and B using the single sample method. Recall from the beginning of this report it was 
suggested that a multiple sample, end-to-end assessment of the precision limit was the 
preferred approach. If this data is available, it is recommended that that approach be 
used. Since multiple samples for the results considered in this report for Test Points A 
and B were not available and the multiple samples for Test Points C-L were 
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demonstrated to contain a systematic variation which might be attributed to the tunnel 
control system, it was determined that the single sample precision limit estimates would 
provide the most reasonable precision limit estimates. 

For these reasons, the single sample precision limits were used, even though 
earlier observations implied that the small (10 points) sample size and short sample 
duration ( 1 sec) may not be adequate. Since the uncertainty in each parameter is 
dominated by the bias error estimate, the selection of the technique to be used to 
estimate the precision limit is not critical for this particular case. The uncertainty in the 
tunnel parameters Mach and Reynolds numbers and angle of attack do not vary 
significantly between the two test points. The non-dimensional force coefficients do 
show considerable differences between the two points. This illustrates the fact that one 
cannot state a single uncertainty for all results over a complete range of parameters. 

As a last comment, one must remember that the uncertainty estimates for the two 
test points were conducted in order to establish a procedure for performing an 
uncertainty assessment in a facility like the NTF. Significant additional effort is required 
before the bias estimates can be refined and the results presented above should not be 
considered as representative of all tests conducted in that facility. 
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Section 6. Conclusions 


This study was conducted in order to establish a framework and identify 
methodology for an end-to-end uncertainty analysis for a wind tunnel system 
using computer based data acquisition and data processing. The methodology 
was based in part upon techniques presented in ANSI and ASME standards 16 
Methods for establishing both bias and precision limit estimates were presented. 
The methods presented in this report were applied to a preliminary analysis of 
measurements of C D and C L at the NASA Langley Research Center National 
Transonic Facility. This preliminary analysis was useful in identifying important 
issues related to calibration experiments, bias limit estimates and data sampling 
procedures. 

The unique characteristics of the type of experiment performed at the NTF 
are the complexity of the operating environment and the large number of 
calibration experiments which influence the uncertainty in the actual tunnel 
measurements. All of the instruments used at the NTF are calibrated on a very 
regular basis. The characteristics of the instrument as well as its uncertainty are 
parameterized by the calibration constants. These calibration constants 
themselves are the results of the calibration experiments and contain uncertainty 
due to the precision of the calibration experiment, uncertainty in the working 
standard used for the calibration and procedures used in the calibration 
experiment. 

It should also be noted that the only ''measurements", in the traditional 
sense of the term, which are conducted in the process of performing both the 
actual experiments and the calibration experiments are performed by an 
electronic circuit which converts "voltages" into an integer number which is 
proportional a voltage difference. The role of electronic sensors and the computer 
in wind tunnel testing has reduced the experimental process into a three step 
procedure. First, physical input to electronic sensors produce voltages which may 
be then subjected to other electronic "processing". Second, computer based data 
acquisition systems convert the voltages to integer, digital form with finite 
resolution. Third, the digital information is combined with the results of other 
experiments and numerical calculations are performed, usually by the computer, 
in order to determine "results." Therefore the uncertainty in the final result is 
intimately tied to the production, electronic processing, conversion, digital 
processing and interpretation of this electronic information. 
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Of particular concern in the experiments conducted at the NTF are the 
strain gage force balance and the angle of attack sensors. In this preliminary 
study these two instruments appear to have the greatest influence on the 
uncertainty in the lift and drag coefficients. The uncertainty is influenced by the 
uncertainty in the calibration constants which are used to process the digital 
representation of the electronic output from these sensors. In order to assess the 
uncertainty in the results determined using these calibration constants one needs 
to quantify the uncertainty in the calibration constants themselves. This is 
particularly important for this type of application where the calibration experiment 
takes place in an environment which is much different from that of the actual 
experiment. This is also influenced by the fact that the electronic data collection 
from these two instruments is quite different in the calibration experiment from 
that in the wind tunnel. Efforts currently underway to establish this type of 
uncertainty information from the calibration experiments will have a very positive 
effect on future detailed uncertainty analyses in this facility. 

The preliminary results presented in this report should be considered as 
just that, preliminary. The uncertainty analysis includes both precision and bias 
limits and the preliminary assessment for the NTF was limited in its attention to 
each. The precision error estimates may be the "easier" of the two factors to 
establish with greater confidence since it can be accomplished by acquiring more 
wind tunnel data. A wider range of operating conditions and test points would 
have to be evaluated to begin to develop a more complete evaluation of the 
precision limit and its primary sources. Due to the cost of operation of the NTF 
this may have to be done as part of existing test programs but it can eventually 
provide insight into the "repeatability" of measurements within the facility. 

Due to the complexity of the NTF system the bias error estimates may be more 
difficult to develop. Only limited bias influences were considered in the current study. 
There are still many issues which should be included in the bias error estimates. Some 
are related to the test environment and others to the instruments and techniques used in 
the measurement process. Some issues that still must be addressed are listed below 
with no special attention to priority. 

- 'gas' modeling and constants 

- flow angularity 

- flow nonuniformity (spatial and temporal) 

- balance temperature gradients 

- loads carried across balance, (i.e. wires, tubes,..) 
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- temperature and pressure sensor installation errors 

- base pressure measurements and corrections 

- other tunnel corrections 

- wind-off zero measurements 

- model assembly and surface irregularities 

- vibrations (model and support system) 

- numerical procedures in data reduction 

Each of these factors influence "numbers" which have a direct impact on the 
results (M, Re, C L ,, C D , etc. ..) of a specific test and therefore they can influence 

the bias error estimates. 

Improvements in the system uncertainty analysis can only be 
accomplished by continuing the process of understanding the "physics" of 
measurement environment and measurement techniques. Since most of the 
decisions made in estimating the uncertainty depend upon the judgment of the 
analyst, it is only through the improved understanding of the experiment that one 
can improve the uncertainty analysis. Uncertainty analysis is an ongoing process 
and must be considered as such to be an effective part of an experimental 
program. 
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Appendix A: Correlated Bias Errors 


When the bias in a result is determined using propagation of errors, it 
should be noted that if the various parameters used in determining the result 
have bias errors which are correlated, the resulting estimate of the bias in the 
result will be altered. There are various sources for correlated bias such as 
calibration against the same working standard. The expression given in Eq. 2.7.2 
is altered if there are correlated bias terms. The bias error in the calculated result 
is determined using, 



X [< e i B Pi> 2+ X e i 9 k PB(k B Pi 
i = 1 k = 1 



BpkO-Sik)] 


Eq. A.1 


where 5 jk is the Kronecker delta ( = 1 if i = k and =0 if i * k )and p Bjk is the 
correlation coefficient between the biases B p . and Bp k . Depending upon the 

"sign" of the correlation term, the bias can be larger or smaller than in the case of 
an uncorrelated bias. 

The following two examples are included to illustrate the influence of a 
correlated bias error for a case where two parameters, x and y, are used to 
compute a result, r. 


r=f (x , y ) 


Eq. A. 2 


Then Eq. A.1 is reduced to, 


B r = 


dr 


1/2 


ar 


( ^ B x) +( ^ B v ) + 2 


0r ar 


ax 


a y r 


Pvv B v b. 


ax a y Kx y x y 


Eq. A. 3 


64 



Since it is difficult to determine the bias in the parameters x and y, it would 
probably be even more difficult to quantify pxy . Therefore in most practical 
applications it is recommended 18 that an effective approach is to rewrite the third 
term in the form, 


3r 3r _3r 3r 

0x a y Px y x y " ax a y 


B X By 


Eq. A.4 


i i 

where B v and B u are computed using only those components of B x and B 
x y y 

that are from the same error source - i.e. they are perfectly correlated so that p xy 

= 1 . The following two cases illustrate the influence of correlated bias errors 

where the result r is either a sum or a difference between two parameters. 


1) if r = x + y and B x = By = B, then, 


1/2 


B, 


5r - - 2 . / ^ D x 2 - ar ar 


‘S 8 *’ +< ^ B y) +2 ax^p X yB* B y J 


1/2 


= [b 2 + B 2 +2Bb] = 2 B 


2) if r = x 


1/2 

= [b 2 +B 2 -2Bb] =0 

This second example is often the source of potential confusion in conducting an 
uncertainty analysis. If the result is the difference between two measurements, as 
might occur in a drag increment test, it is often stated that all bias errors are 
eliminated. This is only true if the bias errors are perfectly correlated. Considering 


y and B x = B y = B, then 

ar 


1/2 


<^ B x> 2+ < J B y> 2 + 2 ^P- B * B 


3r 3r 

di-dy Kx y“ x "y j 
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all of the potential bias error sources, this is probably very rarely the case and 
therefore bias errors will be present, even in "incremental" testing. One should 
never discount the presence of bias errors without a comprehensive uncertainty 
analysis. 
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Appendix B. Bias Estimates Using Computed Results 

In propagation of errors into the calculation of a result particular care must 
be taken to determine the "sensitivities" with respect to independent variables. 
This is illustrated by a detailed discussion in the AGARD Draft and the following 
is included in order to describe the cause of this problem. 

Consider a simple experiment where a and b are independently 
"measured" parameters and there are three results which are computed as part 
of the data reduction process, Q, R and S. If 

R = f (a, b) - a "neat little" expression 

S = g ( a, b) - another "neat little" expression 

Q = F ( R, S) - one more "neat little" expression 


but one could also write: 


Q = G ( a, b) - a more complex expression due to forms of R and S. 


If one wants to determine the bias error in Q, there are a number of approaches 
which might be considered. 

One approach would be to determine the bias errors in the results S and R 
and then propagate those bias errors into the calculation of Q. This appears 
attractive since f, g and F are "easy" functions to work and determining analytic 
expressions for the sensitivities is often one of the more time consuming tasks in 
an uncertainty analysis. This would yield, 


where 


4= (I b r) 2+ (I b s) 2 

b R (I B a ) 2 + (f B b ) 2 


Eq. B.1 


Eq. B.2 
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B?- (^R \ 2 / ^9 □ \ 2 

b S - ' 3a B a' + ( 3b B b) 


Eq. B.3 


substituting Eqs. B.2 and B.3 into Eq. B.1 yields, 


R 2 r/f )2 / 3f . 2 . , IE 2 ,^,2, o 2 

b q ~ i ( aR ) ( a« ) + ( as ) ( ) ] B a 


da 


as > v aa 


+ r ) 2 A 2 + r^)2 ,<^2 1R 2 

+ u 9R > ( db> + ( as } ( ab > ] B b 


Eq. B.4 


An alternative approach would be to compute the bias in Q directly from, 


2 aG 2 / dG p 

b q = (a^ B a) + ( ab B b) 


Eq. B.5 


Now and ^ could be computed by application of the "chain rule" and a 

little algebra yields an expression for the desired bias in terms of the more 
convenient expressions, 






^2 + p dF aR aF as 2 
W + ^aR aa as aa JB a 




(1)2 

w + 


& 2 
w 



aF aR aF as 2 
aR ab as ab J B b 


Eq. B.6 


Comparing Eq. B.4 and Eq. B.6 show that there are "missing" terms in the 
expression for the bias in Q developed using the "interim" results and discounting 
the fact that the R and S are not "independent" parameters. 

One of the purposes of an uncertainty analysis is to provide uncertainty 
assessments in the results of an experiment so that when the results are used in 
subsequent analyses one can estimate their uncertainty. The example shown 
above emphasizes that care must be taken when propagating bias errors or 
uncertainties to make sure that if the uncertainties are correlated, proper error 
propagation methods are used. 
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Appendix C: Bias Estimates: Fixed Parameters 


The following briefly outlines the calculations used to establish the bias error 
estimates for a number of the fixed parameters. This data was collected from calibration 
reports and included to demonstrate the approach taken in this study. It should be 
considered in light of the limited time and data available for this effort. It is not a 
complete list of all the parameters considered. 

1. Balance Primary Sensitivities: Balance NTF113A 

Example: SI - Axial Force (AF) Sensitivity 

SI = .37037 Ibf / p V / V 


a. "Repeatability" 

Consider the variation between the two available calibration experiments. 

calibration date 1/2/92 SI = .3703704 Ibf / pV / V 

calibration date 10/14/93 SI = .370321 Ibf / pV / V 

If this as a sample with two data points and the precision index is based upon the 
difference between the two values, then, 

IASI 1 = 0.000049 Ibf/pV/V 

The precision limit is estimated using the precision index and the "t" factor for one 
degree-of-freedom and a 95% confidence level (t = 12.7), 

P S1 = 12.7 x ( 0.000049 Ibf / pV / V ) 

= 0.000627 Ibf/pV/V 

This is then "frozen" as a bias limit and is a measure of the "repeatability" in the 
calibration process. 

B S1 = P S1 =0.000627 Ibf/pV/V 
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b. Stated calibration accuracy: 


The calibration report for this balance reported a full scale design load for the 
axial "beam" of 400 lb. It also stated a calibration "accuracy" for axial beam of 0.31% FS 
(full scale) with a 95% confidence level. 31 

Uncertainty in axial load = 400 lb x 0.0031 = 1 .24 Ibf 

Actual axial force reading for Test Point A: 

AF= 132 Ibf 

The relative "bias" limit at this value of the balance output would be estimated as: 

B af / AF = 1 .24 Ibf / 132 Ibf = 0.94 % 

Since the primary sensitivity SI is linearly related to the balance output it was assumed 
that one could use the bias in AF as the basis for bias estimate in SI : 

B S1 = 0.0094 x .37037 Ibf / mV / V 
= 0.00348 Ibf/mV/V 

Then the two estimates were combined in "quadrature." 


2 


21 


1 

2 


B S1 = L ( 0.000627 ) + ( 0.00348) 


= 0.00356 Ibf/mV/V 

This produced a relative bias error estimate of 0.96 % for this calibration constant. This 
estimate is dominated by the contribution due to calibration "uncertainty" so that for this 
parameter the repeatability of calibrations was not a significant factor in the bias error 
estimate. 
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2. AOA Constants 


a. Sensitivity - AOA1S (AOA 12-6) 

The stated sensitivity used in the data reduction for the test points considered in 
this study was, 


AOA1 S = 0.270235 volts/g 


Date available on five successive calibrations of a "similar" AOA device (AOA 12-4). 


calibration date 1/4/90 
calibration date 1/11/91 
calibration date 5/13/91 
calibration date 3/5/92 
calibration date 4/20/92 


AOA1 S = 0.268346 volts/g 
AOA1 S = 0.268778 volts/g 
AOA1 S = 0.268853 volts/g 
AOA1 S = 0.268902 volts/g 
AOA1S = 0.268918 volts/g 


Assuming the range (greatest difference between any two calibration points) was an 
estimate of the precision index and t factor of 2.78 (a 95% confidence level with four 
degrees-of-freedom), the precision limit for the calibration data can be determined. 

P AOA1S = 2 - 78x ( 0.000572 volts/g ) = 0.00159 volts/g 


This is then "frozen" and use as an estimate of the bias limit. 
b AOA 1S = P AOA1S * 0.00159 volts/g 

It should be noted that this estimate of the precision index does not account for the 
obvious "trend" in the calibration data. Since the sequential calibrations do not appear to 
yield "random" results, other sources should be considered in making the bias error 
estimates for the AOA sensitivity. 
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b. Bias, Az and Omax 

The values used for these parameters in the data reduction for the test points 
considered in this study were, 

AOA1 B = 0.0002015 volts 
AOA1Z = 47.2746 deg 
AOA1M = 0.32108 deg 

This data was derived from an in-tunnel calibration. Fortunately, three different 
"calibrations" were available for a very similar "test" using a comparable AOA sensor. 
These three calibrations provided additional data for developing a bias estimate for 
these parameters. 


parameter 

AOA IB 

AOA1Z 

AOA IS 

#1 

.0002015 

47.27 

.3211 

#2 

.0002135 

48.33 

.3172 

#3 

.0002405 

46.61 

.3141 

Average 

.0002185 

47.40 

.3175 

Precision Index 

.00002 

0.867 

.0035 

t x Precision Index. 

.000086 

3.73 

0.0151 


The "t" factor ( t = 4.3) was based upon a 95% confidence level and v = 2. 
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Appendix D: Bias Estimates: Measured Parameter 


The following demonstrates the steps taken in order to estimate the bias error in 
one of the measured parameters. This represents the data reduction procedure used for 
one channel of the data acquisition unit and illustrates the process by which "voltage" is 
measured using the calibrated DAU. 

Channel 18 - AXFRAW 
Full Scale Voltage: 10.24 millivolt 
Pre-amp gain: x500 

PGA gain: x2 

This channel was calibrated prior to the actual measurement. This was a "five-point" 
calibration in which a precision voltage source was used to apply a specified voltage to 
the input of the analog-to-digital converter. The average of fifty samples, rounded to a 
whole bit, were used to represent the A/D output. 


Working Standard Voltage 
Source (millivolts) 

A/D Converter Output 
(bits) 

-7.680 

-12262 

-3.840 

-6130 

0.000 

-1 

3.840 

6129 

7.680 

12261 


The bias error in the channel output is due to uncertainty in the working standard, the 
curve fit used for the calibration and the data processing that converts the actual 
measurement to a digital "voltage". 

a. Uncertainty in the Working Standard: 

The bias in the working standard was determined from the manufacturer's 
standards for the precision voltage source and it depends upon the range and 
programmed value for the source. 
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B WS = 3 

+ 0.002% programmed value (7.68 millivolt) 

+ 0.0005% range (100 millivolt) 

= 3 pV + .15 pV + 0.5 pV = 3.65 pV 

b. Least Squares Curve Fit: bits vs. volts 

A least squares curve fit was performed using the data listed in the table on the 
previous page. The procedures outlined in Section 4 were applied to this data. Recall 
that the computer measurement is in "bits" and this digital information must be 
converted to voltage units. This is done using the DAU calibration data for this 
particular channel. The DAU calibration is conducted on a very regular basis, at least 
once a day and often more frequently. This digital representation of voltage is eventually 
used with the instrument calibration information to convert the sensor output to actual 
"engineering units." Using the data from the table an estimate for m and a in the 
following expression was developed using Eqs. 4.2.2 and 4.2.3. 

bits = m * volts + a 

a = intercept = 0 bits 
m = slope = 1596.5 bits/millivolt 

Then computing an unbiased estimate of the standard deviation (SEE) associated with 
the dependent variable, using Eq. 4.2.4, 

a es t = S Yj = 0.948 bits 

The precision limits for the intercept and slope can be computed using Eq. 4.2.5 and 
4.2.6 and the appropriate t factor. Since there were five data points in the fit, a "t" factor 
based on a 95% confidence level and three degrees-of-freedom was used. 

Precision index for intercept = t S a = 1 .35 bits 
Precision index for slope = t S m = 0.25 bits/p V 

These precision limits are then "frozen" as the bias limits due to curve fitting. 
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c. Data "Reduction": 

Once the curve fit is completed, the individual measurements are processed using the 
following formula. 

VOLTS = M * BITS + A 

where, 

A = - a / m =0 volts 
M = 1 / m = .62638 |iV / bit 

and "a" and "m" are the result of the least squares curve fit. 

The bias limits for the parameters A and M were determined using Eqs. 4.3.7 and 4.3.8 
and the bias limits for a and m determined from the curve fitting process, 


1 



= 8.45 x 10 ' 7 volts 


and: 

g 

B M = — = 9.73x 10 ‘ 14 volts/bit 
m 

Finally as an example, for channel 18 at Test Point A the DAU output was 1782 bits (i.e. 
BITS = 1782). This was the result of an average of 10 data points sampled during a 1 
second interval and rounded to the nearest whole integer. 

VOLTS (AXFRAW) = .62638 |iV / bit x ( 1782 bits) + 0 volts = 1 .1 1619 millivolts 

This represents the basic measurement of the output of this instrument. This process is 
repeated for every data point "taken" using this channel. 
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d. Estimated Bias Components for Voltage Reading: 

The bias estimated on the measured voltage will depend upon the voltage level, 
the uncertainty in the curve fit, and working standard. 

1 . Bias estimate on DAU output = 1 bit 

This is based upon the "least count" resolution of the A/D converter. 

2. Bias estimate on WS = 3.65 pV 

This was determined from the manufacturer's specification as shown. 

3. Bias estimate on the intercept (A) 

This is a statistical assessment of the curve fitting process. 

Ba= [( b 'W« f + ( b a ws ) 2 ] 2 > 5 

= 3.75 fiV 

4. Bias estimate on the slope (M) 

This is a statistical assessment of the curve fitting process. 

B M = [( B M cu _ m ) 2 ] 2 > * 

= 9.73x 10 ' 14 volts/bit 


e. Bias Estimate for Voltage Measurement: 

These are combined to provide the voltage bias limit estimate 


B VOLTS = 


2 2 
( B A ) + ( BITS * B m ) + ( M 


2 

b bits) 


1 

2 


= 3.8 [iV 
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Then for this data point, including bias errors only, the "measurement" can be 
interpreted as: 


AXFRAW = 1.116 ± .004 millivolts 

The bias limit is 0.34 % of the reading and it is dominated by the uncertainty in 
the working standard. This procedure was then repeated for each analog input 
channel. 
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Table 1. Relative Sensitivities, Test Point A 
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Table 2. Relative Sensitivities, Test Point B 
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Table 3. Parameter Bias Error Estimates - Test Point A 
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Table 4. Parameter Bias Error Estimates - Test Point B 



Parameter 

Bias Factor 
Mach Number 



PS50V 

1 .55 E-07 

PT100V 

1.39 E-07 

beta 

9.40 E-08 

lambda 

1.31 E-09 


Table 5. Prioritized Bias Factors, Mach Number, Test Point A 


Parameter 

Bias Factor 
Reynolds Number 



beta 

| 6.02 E-06 

lambda 

3.03 E-06 


2.66 E-06 

PT100V 

2.56 E-07 

PS50V 

4.84 E-08 

RO 

3.08 E-08 


Table 6. Prioritized Bias Factors, Reynolds Number, Test Point A 


Parameter 

Bias Factor 
Angle of Attack 



AOA1B 

3.94 E-04 

AOA1Z 

2.34 E-04 

AOA1M 

1 .45 E-04 

AOA1S 

1 .88 E-05 

ELEVA 

9.80 E-07 


Table 7. Prioritized Bias Factors, Angle of Attack, Test Point A 
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Parameter 

Bias Factor 
Drag Coefficient 



SI - AF 

1 .25 E-04 

AXFRAW 

2.34 E-05 

S3 - NF 

5.30 E-06 

AOA1Z 

Mill! HIM 

S5- PM 

1.81 E-06 

AOA1M 

1.61 E-06 


6.83 E-07 

AOA1B 


PT100V 

5.40 E-07 

PS50V 

2.42 E-07 


Table 8. Prioritized Bias Factors, Drag Coefficient, Test Point A 


Parameter 

Bias Factor 
Lift Coefficient 



S3 - NF 

1.31 E-03 


1 .28 E-04 

NF2RAW 

3.70 E-05 

WNF1 1 

6.04 E-07 

S5- PM 

5.98 E-07 ] 

PT100V 

5.35 E-07 

BALPOW 

2.65 E-07 

PS50V 

2.35 E-07 

beta 

1.76 E-07 


Table 9. Prioritized Bias Factors, Lift Coefficient, Test Point A 
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Parameter 

Bias Factor 
Mach Number 



PS50V 

1 .56 E-07 

PT100V 

1.41 E-07 

lambda 

3.31 E-10 

alpha 

5.95 E-1 1 


Table 10. Prioritized Bias Factors, Mach Number, Test Point B 


Parameter 

Bias Factor 
Reynolds Number 



beta 

4.56 E-06 

lambda 

2.94 E-06 


2.65 E-06 

PT100V 

2.59 E-07 

PS50V 

4.92 E-08 

RO 

3.09 E-08 


Table 1 1 . Prioritized Bias Factors, Reynolds Number, Test Point B 


Parameter 

Bias Factor 
Angle of Attack 



AOA1B 

6.72 E-05 

AOA1Z 

4.18 E-05 

AOA1S 

4.17 E-05 

AOA1M 

2.58 E-05 

ELEVA 

1 .74 E-07 


Table 12. Prioritized Bias Factors, Angle of Attack, Test Point B 
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Parameter 

Bias Factor 
Drag Coefficient 



S3-NF 

MHIIHMI 

AXFRAW 

7.30 E-05 | 

SI - AF 

— 1 1 ■ ii i 

AOA1B 


AOA1Z 


AOA1S 

WHAHi 

AOA1 M 1 


NF2RAW 

4.32 E-06 

PT100V 

5.47 E-07 

S5- PM 

3.37 E-07 

NF1 RAW 

■ II !■ IN M 

PS50V 

2.45 E-07 

ELEVA 

1 .02 E-07 


Table 13. Prioritized Bias Factors, Drag Coefficient, Test Point B 


Parameter 

Bias Factor 
Lift Coefficient 



S3 - NF 

1 .94 E-04 

NF2RAW 

■ 



PT100V 

5.43 E-07 

PS50V 

2.38 E-07 

! BALPOW 

4.29 E-08 

beta 

1 .49 E-08 


Table 14. Prioritized Bias Factors, Lift Coefficient, Test Point B 
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Table 15. Bias Limit Estimates for Results - Test Point A 


Parameter 


Nominal Parameter 
Value 


Bias Limit 
Estimate 



0.798 

± 0.0004 

39.91 x 10 6 

± 0.13 x 10 6 

2.18 0 

± .03 0 

0.0220 

± .0004 

0.404 

± .0067 


Mach Number 


Reynolds Number 


Angle of Attack 


Drag Coefficient 


Lift Coefficient 


Table 16. Bias Limit Estimates for Results - Test Point B 
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Table 17. Precision Limits, Measured Parameters, Test Point A 








Parameter 

Precision Factor 
Drag Coefficient 



AXFRAW 

3.47 E-06 

NF1 RAW 

3.53 E-07 

NF2RAW 

1 .49 E-07 

PS50V 

7.23 E-08 

ELEVA 

5.12 E-08 

PT100V 

4.15 E-08 

BALPOW 

1 .43 E-09 


Table 18. Prioritized Precision Factors, Drag Coefficient, Test Point A 


Parameter 

Precision Factor 
Lift Coefficient 



NF1 RAW 

6.63 E-05 

NF2RAW 

5.13 E-05 j 

PS50V 

7.01 E-08 

PT100V 

4.11 E-08 

BALPOW 

1 .63 E-08 


Table 19. Prioritized Precision Factors, Lift Coefficient, Test Point A 
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Parameter 

Nominal Parameter 

Precision Limit 


Value 

Estimate 




Mach Number 

0.8015 

±0.0002 

Reynolds Number 

40.056 x 10 6 

±0.007 x 10 6 

Angle of Attack 

-0.921 0 

± 0.004 ° 

Drag Coefficient 

0.01554 

±0.00003 

Lift Coefficient 

0.0351 

± 0.0004 


Table 20. Precision Limit Estimates for a Single Result - Test Point A 


Parameter 

Nominal Parameter 

Precision Limit 


Value 

Estimate 




Mach Number 

0.7982 

±0.00015 

Reynolds Number 

39.913 x 10 6 

±0.005 x 10 6 

Angle of Attack 

2.184° 

± 0.004 ° 

Drag Coefficient 

0.022198 

±0.00015 

Lift Coefficient 

0.404 

±0.002 


Table 21 . Precision Limit Estimates for a Single Result - Test Point B 
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Table 22. Precision Limit Estimates, Multiple Test Results, Test Points C - 






























Mach No. 

Reynolds No. 

Alpha (deg) 

C D 

c L 

■ 






Nominal 

Parameter 

Value 

0.801 

4.01 E+07 

-0.92 

0.0156 

0.0351 







Bias Limit 

0.0005 

1.40E+05 

0.03 

0.0002 

0.0013 

Precision Limit 

0.0002 

7.00E+03 

0.004 

0.000032 

0.00038 







Mil 1 III Hit I 1 — 1 

±0.0005 

± 1.40E+05 

±0.03 

±0.0002 

±0.0014 

Confidence 

Level 

95% 

95% 

95% 

95% 

95% 


Table 23. Uncertainty Estimates, Single Measurement Method, Test Point A 


Parameter 

Mach No. 

Reynolds No. 

Alpha (deg) 

CD 

Cl 







Nominal 

Parameter 

Value 

0.798 

3.99 E+07 

2.18 

0.0220 

0.404 







Bias Limit 

0.0004 

1.3 E+05 

0.03 

0.0004 

0.007 

Precision Limit 

0.00015 

5.0 E+03 

0.004 

0.00015 

0.002 








± 0.0004 

±1.3 E+05 

± 0.030 

±0.0004 

± 0.007 

Confidence 

Level 

95% 

95% 

95% 

95% 

95% 


Table 24. Uncertainty Estimates, Single Measurement Method, Test Point B 
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Figure 1. Schematic of Uncertainty Analysis Methodology 
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b. "Short" Sample Inverval 


Figure 2. Time History of Random Data Sample 
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Figure 3. Basic Measurement System Schematic 
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Figure 5. NTF DAU Schematic 
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Figure 6. NTF DAU Details 
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Figure 7. DAU Calibration, In- Tunnel 
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Figure 9. Ruska Two-point Calibration, In-Tunnel 
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Figure 10. Ruska Pressure Measurement 
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Figure 1 1 PRT - Calibration Laboratory 
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Figure 12. PRT Total Temperature Measurement 
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Figure 13. Single Axis AOA, Calibration Laboratory 
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Figure 14. Single Axis AOA, In-Tunnel Calibration 





Figure 15. Single Axis AOA, Orientation Measurement 
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Figure 17. Force Balance Calibration in Model Preparation Area 
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Figure 18 . Force Balance, In-Tunnel Force and Temperature Measurement 
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Figure 19. Calibration Experiment and Measurement Schematic 
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Figure 20. Test Point A, Sample Time History, AXFRAW 
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Figure 21. Test Point A, Sample Time History, PTV100 
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Figure 24. Drag Coefficient vs. Angle of Attack, Test Points C - L 
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